Looking for something to replace NetApp FAS2020A for VMware storage

H

HDClown

Limp Gawd
Joined
Nov 30, 2004
Messages
222
I am shopping for a replacement for a NetApp FAS2020A that will be out of space soon. I only have the single active/active controller shelf with 450GB drives and it will hit it's 3 year mark in a few months, so I'm just going to retire it down to non-production level storage. So, I'm looking for replacements.

Environment consists of 3 vSphere 5 hosts (DL380 G7) with 40 VM including Exchange 2010, SQL 2008 R2, SQL 2012, and SharePoint 2010. From a 3 year growth standpoint, I MAY add a 4th host, or refresh the 3 hosts I have. I am greatly under-utilized on CPU and a recent RAM upgrade has me in a good place here. I could actually run the entire environment on 1 host if I needed. I have 3 so that if I need to down a host for an extended period, I still have redundancy.

Some performance info:

IOPS During production hours
-1000 @ 95th percentile
-1400 @ 99th percentile
-2000 @ peak
-About 50/50 on read/write with lots of read bursts

IOPS with non-production hours (all storage, backup window now included, hence higher numbers)

- 1800 @ 95th percentile
- 2500 @ 99th percentile
- 3000 @ peak
-Weighted much more towards reads

Note that these IOPS are based on Exchange 2003. I'm currently migrating to Exchange 2010 which will reduce some IOPS, however, we are adding more SQL workload which will increase IOPS (more read than write). There are unknowns on the new SQL workloads, but I do not expect anything drastic, so I'm forecasting for growth up 1500 IOPS @ 95th percentile during production hours and still staying around 2000 IOPS during backup windows.

Due to cost limitations, I cannot go 10GbE so I'm looking at 4GbE on the storage network which will include new switches. I don't have a final decision on those switches (2x24 port) but for cost reasons, I'm looking at Del PC7024 or Force10 S25, and I'm leaning towards the Force10 right now as I will want to replace existing HP2910al core switches in the future and Force10 will have nice options allowing me to keep it homogenous.

Solutions proposed from vendors have been EQL PS6100X with 24 600GB discs and NetApp FAS2240-2 with 24 600GB discs. The NetApp solution would $10k more than the EQL. Before I got my 2020A I looked at HP P4000 so I'm also looking at the new generation stuff, StoreVirtual 4730. This sizing came based on two things 1) IOPS, 2) future storage needs. I currently use about 2.5TB of storage and with some things we will do with Exchange, and new SQL workflows, I'm looking at being at about 4TB within 12 months, so I had a minimum target of 6TB. I could be there will 300GB disks but after looking at price, it seemed to make sense to go to 600GB and have more growth room.

From a DAS standpoint, I've also looked at Dell MD3200, HP P2000 G3 MSA and IBM DS3500. With the way these price, I could get 2 of these for the cost of the EQL or NetApp and due replication between then for more redundancy than EQL/NetApp budget affords me.

I'm also open to VSA. I can put drives in my hosts, upgrade to a P420 and use SmartCache and have even more space for less money, and gain even more redundancy of hardware as compared to a single unit of something above.

I want something with IOPS headroom and something that won't be depreciated within a couple years (I'm feeling burnt by the FAS2020A situation given that It's just not hitting 3 years in-use). Of course, I won't something that can scale and not cause me to have to re-do this all over again in 2-3 years.

I have Veeam B&R Enterprise that I use for Backup and Replication. I don't do any NetApp level snapshots now due to space issues but it would be nice to be able to do this for more frequent restore points in-between backup and replication windows, BUT, this is not a requirement.

Last thing here since I know it's hugely popular, I do not want a ZFS solution, be it white box or supported model. I have zero experience with it and I'm not jumping into it for primary SME storage in this deployment.
 
Last edited:
Ah yes, Nimble. I signed up for their mailing list last year and get e-mails all the time but mostly ignored them until recently. They seem to be quite the rage now. NetApp offers 18 SATA + 6 SSD option on the 2240-2 recommended by my VAR but I don't know how it prices compared to 24 10K SAS or a Nimble CS220 (I don't know real world price on a CS220 for that matter either) so I will get some of that pricing today hopefully. Also need to get the unbiased opinion of NetApp VST/Flash Pool's vs Nimble's approach.
 
Here are some discussions about newcomer Tegile:
http://www.theregister.co.uk/2012/06/01/tegile_zebi/
- Washington and Lee University implementing 500 virtual desktop images and doing some file-sharing. The university achieved 7X the IOPS of its previous storage and a 70 per cent reduction in VDI capacity needs.
- Virgin America uses Zebi in a virtual server scenario and found it was three times the speed of NetApp kit at a tenth of the price; at $875,000 vs an $80,000 Zebi box.
- Starwood Capital Group recorded a dual controller Zebi config delivering 180,000 IOPS which was six times the speed of an EMC Clariion CX for v15 per cent of the price


And another discussion by sysadmins in VMware thread:
http://communities.vmware.com/thread/398035
"Not sure what your requirements were, but every interaction I’ve had with Tegile has been fantastic. I admit that they did not have a bunch of pretty documentation to send me. I asked a lot of questions on first contact and got the answers needed to interest me enough for a webex demo of the management interface, which looked good"

"I also talked to Tegile during the Citrix conference in San Fran this year and was impressed by the product, but moreso by a starting price of 16k."

"My experience with Nimble directly was not so great. Sales was nice up front, but as soon as it looked like I might go with Tegile the sales rep immediately resorted to trying to guilt me into their product by telling me he had already gone to his boss for pricing approval and so on. I talked to others who had a similar experience. This is unfortunate because Nimble obviously has a good product, but bad attitudes can spoil many things."

"I have moved most of my servers into our VMWare environment, along with 8 customer service and 2 dev VDI guests. The Tegile unit is serving all of it without breaking much of a sweat at all."
 
VNXe would work but it is the most expensive option compared to everything else, so I've knocked it off the list because of that.
 
Are you going with an HA 2240-2? You lose 3 drives to the other controller for ONTAP if you have an HA pair. Are you using A-SIS? I'm not sure how they're doing 18+6 in a 2240-2. IIRC, it would have to be a 2240-4. Their SSD's are 3.5 inch drives along with their SATA. That's a 4243 shelf, much larger. Not a big deal but just an FYI. Quick calculation using my netapp spreadsheet shows if you went non-HA you'd get about 8TB worst case with overhead using 600GB drives. That two RG's of 11 drives and 2 hot spares. You could get away with 1 hot spare if you really wanted. On something that small I wouldn't do a dedicate root aggregate/volume.

Equallogics have dual controllers but are not active/active. Require more connections, standby controllers connections are unused until needed. Each shelf you buy requires at least 4 connections for both controllers but another NetApp shelf only needs SAS connections to the head.

I have both in my org but we're replacing the Equallogics with NetApp. Don't get me wrong, there's a penalty for using NetApp, the loss of 3 disks for a root volume on the second controller blows unless you have different disk types and allow one to control one type and the other to control the other. Since you have experience with NetApp there's not much you'd have to learn. Although the Equallogics are blindingly simple.
 
Yes to HA config, well aware of the space hit with NetApp. That's where they swoop in and tell you "we have de-dupe so you will gain it back, no problem". Well, I have de-dupe on the 2020A (12x450GB) and I only see about 400GB worth of de-dupe, so I'm not getting all of the space penalty back, but it's better than zero back. Fortunately for the environment, it's not 24/7 production so de-dupe can run in the evening. However, as I understand it, if you use SnapMirror between 2 NetApp units, the data will get re-hydrated before the Snap, which mean need a balloon factor of space if doing such a thing, kind of stinks.

The 18+6 was a 2240-4. That config with 100GB SSD and 1TB SATA drives is about $4k more than the 2240-2 with 24x600GB disks. I don't know how that splits up across controllers though so now idea on how that total disc compared. The 3TB based config is $20k more than 24x600GB discs, and there is no 2TB config.

Running dual controller NetApp filer in an active/passive type config where you have to dedicated a minimum of 2 disks (hot spare is not required) for root volume on an unused controller is a big waste of disk, and it's still not 100% foolproof in this config for NDU or failover as compared to active/active, which is how they are designed to run.

The thing I like best about NetApp is NFS. NetApp's are NAS devices, the SAN block storage is just add-on. NFS is nice and easy, works great in VMware, but, I've had VMware Support tell me that iSCSI provides some safeharbor for getting your data if you get in trouble with wacky storage things, where as with NFS, it's all handed off to the NAS and it's out of their control. That being said, iSCSI is more complex, although I don't find there to be anything overly complex about it personally. The thing I don't like about NetApp is the high add-on software cost. Compared to someone like EQL, where it's 100% included, that's a big cost swing.

What I like about EQL is it's built as scale-out, which I'm a fan of scale-out vs. scale-up for SAN, even with the extra cost. Cluster Mode with ONTAP 8 is there, but it's still pretty new. IME, it's rare to fund yourself not needing both at the same time, unless you've done some really poor projections for growth prior to purchase. The biggest downside for EQL is 15MB block size, which kills low bandwidth based replication. It's easy to tackle with WAN optimizers like Riverbed, which I happen to have, so it's not really a big issue.

I'm taking a hard look at Nimble, as it's risen to the top. I'm also taking a hard look at HP StoreVirtual VSA given that I have 3 2U servers with completely empty drive bays, and each chassis can hold 16 disks.
 
The FAS2020 is super slow. I have 2 of them for light workload VDI and every time deleting some VMs, CPU is at 100%.

I have almost all of the storage system you mentioned, by far I still like NetApp. I have FAS 2040 and 2240 for remote sites, 6280 in primary datacenter and 3240/3270 in secondary datacenter. The 2240 is a work horse for remote site but for some small organizations, it could be datacenter level.

Due to cost cut, I have to look for other solutions as well for our remote site deployment from now on. NetApp is too expensive. I have Nimble (not a big fan of it) for VDI now to replace FAS2020, but we need NFS so there aren't many options there. I'm looking at Tegile as well and I'll have a meeting with them next month.
 
Last edited:
The purchase of a FAS2020 was ultimately ill-advised. I was the one who made the purchase but I relied heavily on the specialists at CDW and they assured me it was adequate for my needs. I've come to learn since then over the past few years that this the 2000 series was an area NetApp would love to have never gone through, but what's done is done. It's been adequate on performance for my needs today, but will not be into the future.

The biggest issues is that it's dead product, from the filer to the shelves. I can get some good deals on 1 year off-lease hardware that has been sitting in storage, about $3600 for a DS14MK2-AT with 14 1TB HDD's and 1 yr warranty or $4400 with 3 year. I can build an entire custom solution with way more space for less, so not sure what to do. This unit is going to be backup/replication target once replaced.

2240 should be adequate for my needs WITHOUT VDI. It won't cut it for VDI, even with a mix of SSD/SATA and Flash Pool. Really need 3200 series and Flash Cache, or be able to just buy SSD to run VDI on, but that gets expensive compared to options like Nimble, Tegile, and Tintri as hybrid ground-up solutions.
Can you elaborate on what you don't like about Nimble? I don't need NFS as I'm fine with iSCSI, and I don't' need CIFS either for that matter (another big seller for NetApp)

If you need NFS, NetApp is really the king. Anything else will be block level storage with some kind of add-on, usually a gateway device, to give you NFS.
 
You dont have to dedicate 3 drives for the root vol on a 2240 it can sit in the main agr of data (not best practice but afaik its supported), and depending on your load on the 2020 you have now you may be able to do an active passive.

Do the machines besides VDI need flash pool?

No snap mirror for backup for you if that was your plan, I dont believe the 2020 can do 8.1 and if you use Snapmirror the dr has to be at the same or higher lvl of ontap. Makes me mad as I am in a similar boat with my FAS 270s

I love NetApp NFS especially with the VSC software in ESX. I only do block for exchange and sql.

Have you looked at local cache for your VDI server?
 
My root vol on the 2020A sits on the main aggr and this was the original provisioning of aggr and vols as done by NetApp outsourced implementation services too. I would do the same on the 2240A. Dedicating 3 disks to just the root vol would be a massive waste of space and cash in my case since I won't have a huge amount of disks.

Flash Pool would only be necessary for VDI. I've done some more research on Flash Pool and if I understand correctly, the SSD aggr is used for copies of data on the main aggr, so you don't actually gain more space. The SSD aggr has to bet he same RAID level as the main aggr, so with RAID-DP and active/active controller, I'd end up with 12TB of raw space vs 12.6TB raw space with 24x600GB config, so usable space is no different with 1TB drives in the SSD/SATA mix (which is the same price range as 24x600GB drives).

So there is no space benefit. I also understand that the Flash Pool caches an entire aggregate for which it is assigned, so it's always going to be caching the SATA aggr's. That was essentially the idea as SATA disks have lower IOPS than the 10K SAS disks, so the Flash Pool idea was to help make up for that as a whole, while also being able to assist with VDI. Please correct me if I'm wrong n any of the Flash Pool information.

I was not informed of the SnapMirror issue you mention and I just read the SnapMirror considerations which confirm it. That certainly eliminates one of the major reasons CDW is proposing staying with NetApp. This puts me in more doubt of what to do with the 2020A. Putting more money into it for it to be strictly an expensive backup target for Veeam seems silly. The $3600 I'd pend for 14x1TB HDD's could buy me something like a Synology RS2212RP+ and 12 3TB HDD's giving me way more space, or I could buy a Norco/SuperMicro shelf and put my own gut in and even more HDD's. It seems like my best option for the 2020A at this point is going to be simply to hold my test/dev environment, which was originally going to sit with my new production storage.

I really like NFS from simplicity standpoint. I run100% over NFS for a KISS approach. I never found any really good reasons to do block for Exchange/SQL, be it separate LUNs presented to the hosts, or LUNs directly inside the VM.

As far as VDI, we may not end up needing it as a combination of SSL VPN w/2FA and RDS or XenApp may end up being adequate, so that would keep me away from the much heavier IOPS of VDI, which does simplify things a little. What are you thinking with local cache for a VDI server. If storage is on a SAN, how would the local cache be able to interface out to the SAN?
 
HDClown said:
I run100% over NFS for a KISS approach. I never found any really good reasons to do block for Exchange/SQL, be it separate LUNs presented to the hosts, or LUNs directly inside the VM.
Click to expand...

For a smaller environment, sure I get it. But Exchange databases on NFS is definitely not a MS supported configuration. SnapManager for Exchange and SQL only work with SnapDrive and that needs iSCSI. Everything else can be NFS though. Also, if you have an active/active controller, you're going to have to dedicate 3 or 2 drives for the second controller's root volume. No way around it.
 
Last edited:
QHalo said:
Also, if you have an active/active controller, you're going to have to dedicate 3 or 2 drives for the second controller's root volume. No way around it.
Click to expand...

Are you saying this because your thinking of an active/active deployment where most disks on are Controller1 and Controller2 only has disks to run it's root volume? That's the only way I can see this statement making sense. I split my disks across my active/active controllers and the root vol on both controllers is on the same aggregate as other vols, there is no requirement for it to be separate, at least with ONTAP 7.2/7.3

QHalo said:
For a smaller environment, sure I get it. But Exchange databases on NFS is definitely not a MS supported configuration. SnapManager for Exchange and SQL only work with SnapDrive and that needs iSCSI. Everything else can be NFS though.
Click to expand...

I could go on for hours on what's "supported" with Exchange. VMware has battled MS on this for years because VMware has had support for things that Hyper-V hasn't for many years. The fact that Hyper-V storage had to be local disk, iSCSI or SAN has driven a lot of MS requirements. Now Hyper-V with Server 2012 lets you put VMDK's on SMB 3.0 shares, and things are changing in terms of MS requirements.

MS has some articles that say only FC is supported with DAG, not even iSCSI. There is inconsistency all over the place in Microsoft's own material. If you strictly follow every MS spec/requirement you find, you'll end up pulling out all your hair and abandoning virtualized Exchange all together, going back to physical server with local disk.

When it comes to virtualization and Exchange and storage, it all has to be "block level". Beyond that, pick whatever the hell you want to use and make sure it meets performance requirements and that whatever you choose meets the requirements of any SAN/NAS integrated software you want to use. A VMDK sitting on a NAS connected over NFS, when added to the virtual guest, is still block level storage.
 
HDClown said:
Are you saying this because your thinking of an active/active deployment where most disks on are Controller1 and Controller2 only has disks to run it's root volume? That's the only way I can see this statement making sense. I split my disks across my active/active controllers and the root vol on both controllers is on the same aggregate as other vols, there is no requirement for it to be separate, at least with ONTAP 7.2/7.3
Click to expand...

Yes I am. I have a FAS2220 with only 12 disks at my remote sites. I just put 3 disks for the second controllers root vol there and the rest resides with the first controller. So technically not an active/active configuration but still provides redundancy in the case of controller failure. With 24 disks I get it that you may want to split that up but that also means losing another two disks to parity using both controllers. You're damned if you do and you're damned if you don't I guess.

I could go on for hours on what's "supported" with Exchange. VMware has battled MS on this for years because VMware has had support for things that Hyper-V hasn't for many years. The fact that Hyper-V storage had to be local disk, iSCSI or SAN has driven a lot of MS requirements. Now Hyper-V with Server 2012 lets you put VMDK's on SMB 3.0 shares, and things are changing in terms of MS requirements.
Click to expand...
It's only changing because MS didn't create NFS, they created SMB3.0. Of course they would support it. It only makes sense.

MS has some articles that say only FC is supported with DAG, not even iSCSI. There is inconsistency all over the place in Microsoft's own material. If you strictly follow every MS spec/requirement you find, you'll end up pulling out all your hair and abandoning virtualized Exchange all together, going back to physical server with local disk.

When it comes to virtualization and Exchange and storage, it all has to be "block level". Beyond that, pick whatever the hell you want to use and make sure it meets performance requirements and that whatever you choose meets the requirements of any SAN/NAS integrated software you want to use.
Click to expand...
I don't see all the inconsistencies from MS, but from vendors. NetApp would have you believe that putting Exchange on NFS is perfectly acceptable in certain docs, then their best practice guide has you using SnapManage for Exchange which needs block. But MS has clearly stated that NFS is not a supported configuration. I'm not sure how cut and dry it really can get. Honestly, it boils down to whether you call MS for support of which with such a small environment I doubt you do so use whatever you want. Having MS actually ask you what the underlying storage protocol is if you call them is highly unlikely but for those of us with larger environments, adhering to supported configurations means wading through all of this and ultimately using MS' documentation.

MS Documentation said:
The storage that is used by the Exchange Server guest machine for the storage of Exchange data (for example, mailbox databases or hub transport queues) can be virtual storage of a fixed size (for example, fixed virtual hard drives [VHD] in a Hyper-V environment), SCSI pass-through storage, or Internet SCSI (iSCSI) storage. Pass-through storage is storage that is configured at the host level and that is dedicated to one guest machine. All storage that is used by an Exchange guest machine for the storage of Exchange data must be block-level storage. Exchange 2010 does not support using network attached storage (NAS) volumes. NAS storage that is presented to the guest as block-level storage by using the hypervisor is not supported. Pass-through volumes must be presented as block-level storage to the hardware virtualization software. This is because Exchange 2010 does not support using network attached storage (NAS) volumes. The following virtual disk requirements apply to volumes that are used to store Exchange data.
Click to expand...

Seems pretty cut and dry to me.

A VMDK sitting on a NAS connected over NFS, when added to the virtual guest, is still block level storage.
Click to expand...
No it's not, its a file on NFS.

Sorry if the thread has derailed. :/
 
Last edited:
Look, all I'm saying is, there's nothing technologically wrong with Exchange data with VMware/NFS. It works, with no issues, at all sizes, with 100% functionality maintained (at least if we're talking NetApp as the back end). Being supported or not is unrelated to if it will work or not.
 
If you take Microsoft's official support stance on Exchange at face value then no company in the world that uses Netapp runs Exchange on it.

Their official support stance is crazy. How it is implemented it apparently much more sane.
 
I personally have not been very happy with the Dell EQL line in regards to performance, reliability, flexibility of storage and in the end support.

Remember in all of your quotes to make sure that they're giving you 3 years of full 24x7 4hr turn-around coverage, gives you the best apples-apples breakdown.

One of the great things about Netapp, support is top notch. From drive failures to major hardware issues, simple upgrade questions to major "oops I didn't mean to do that" screw ups they got you covered.

I realize it may not be in the budget but I highly suggest no matter what platform you go with you with a HA setup, so dual heads for the controller or whatever solution you end up choosing, make sure it's HA.

I realize you may not have previously considered ZFS but Oracle just put a bunch of pressure on their sales guys to bust out this quarter so you should at least get a competing quote from Oracle for a ZFS setup equal to what you were looking at with Dell and Netapp. Even if you're not familiar with ZFS you don't need to be as the filer does the bulk of the work for you.

From the IOPS you're looking for a 2240 will treat you well and will cover you from many different sides. VM storage, NFS, CIFS, db luns... you're covered and you have a simple interface to work with once it's setup.

Again, I know you didn't want to even think about ZFS in any form but you really should give the folks at O a call and have them make you a killer deal, it's a do it all fairly well platform similar to Netapp but better than EQL or HP's stuff IMO. That and they're under the gun for making their sales figures this quarter so you could get a smoking deal.

I personally would steer clear of DAS in any form, HP, IBM, Dell, whoever. It's a dead end road with horrible drive failure support and I don't have time for that in my line of work.

Also, since at least exchange 2010 you have full support and well documented best practices using Exchange in a HA VM environment using Netapp storage even in a geographically diverse DR/failover environment.

I think you'll be a lot happier with the Netapp solution in the short and long run, their background and history with a proven software suite, thin provisioning that works, etc... is really ahead of the rest in many regards. yes they cost more, but there is a reason for that. SnapMirror, SnapVault, Cloning, Restoration, etc, etc, etc... are all incredible products and they really work well. The Netapp dedupe works exceptionally well in VMware environments, keeping most of the critical/core files in running memory/cache. If you want it to perform even better add a flash cache device and it just gets crazy good performance.

Let me know if I can help out in any other way.
 
You must log in or register to reply here.
Back
Top