Putting the VM's on a NAS

T

Tolem

n00b
Joined
May 27, 2010
Messages
33
Right now all of my hyper-v and esx-i VM's are on internal disks (10,000 RPM wd raptor).

Will I lose much performance by moving them to my raid 6 array? The box has 4 gigabit nic's and it only uses about half that at the moment.

Or how do you usually have your VM's stored / attached to the server?
 
Network bandwidth will not be the bottleneck. What disks are in there?
 
10k RPM western digital raptor judging by hist first post is what he plans to use for hard drives ;)
Still leaves the question of how many does he have/plans to use?
 
Even so, lopo is right. The random small reads and writes, and IOP requirements will be what kills it.
 
Last edited:
The raptors are whats in the VM server right now, im currently running the VM's off those in JBOD fashion sata 3.0

But what I was thinking of doing was moving the VM files to the NAS array which right now is 10 of the hitachi deskstar 7200rpm drives in raid 6. That way I could have more frequent backups of the VM's, have more space for future VMs, repurpose the raptors and simplify the VM machine down to just an OS drive.

But if it was going to result in slower VM performance I did not want to do it.
 
Does your NAS support iSCSI? if not the overhead of CIFS if it is a simple mapped drive may kill it performance wise.
 
if you can I would put them on the VM host server. However, if you have a farm, and want a central location for your VMs for the purposes of easier backup then I can understand a NAS
 
I was planning to use iSCSI as my NAS is running server 2008.

I've never seen a VM host server before?? Is it going to be much faster?? The reason I would like to use the NAS was to avoid the costs associated with building another server, and because I think the NAS has enough free resources to handle the load as long as the speed will be the same I'm all for it
 
Sounds like your NAS is being used for other functions. I would stick to a dedicated storage pool for your VMs unless you know for certain that your workload and IOPS requirements can be met by the existing NAS>
 
Shockey said:
10k RPM western digital raptor judging by hist first post is what he plans to use for hard drives ;)
Still leaves the question of how many does he have/plans to use?
Click to expand...

well, that depends - is he going to relocate all of those to a NAS appliance?
 
Tolem said:
The raptors are whats in the VM server right now, im currently running the VM's off those in JBOD fashion sata 3.0

But what I was thinking of doing was moving the VM files to the NAS array which right now is 10 of the hitachi deskstar 7200rpm drives in raid 6. That way I could have more frequent backups of the VM's, have more space for future VMs, repurpose the raptors and simplify the VM machine down to just an OS drive.

But if it was going to result in slower VM performance I did not want to do it.
Click to expand...

Significantly lower performance. Especially in RAID6, if you have a lot of writes.
 
lopoetve said:
Significantly lower performance. Especially in RAID6, if you have a lot of writes.
Click to expand...

BBWC goes a long way though. A 10x Hitachi array would actually write decently fast if you are writing from raid card cache. IOPS might be OK since you are adding a ton of spindles. The big issue with raid 6 is you write two parity blocks plus a single written block to the same array. Raid 5 is 1 write 1 parity. Raid 1 is essentially 1 write (well it does two but different disks). Oversimplified, a bit, but with more spindles more cache, and more RAM you can have a pretty decent raid 6 setup.
 
Depends on what you call "decently fast". ;) Will it help? Definitely - especially for spikes. But sustained?

IOPS are IOPS, and cache only buys you so much. Once you approach the low watermark, you're dumping data to disk as fast as the spindles can take it (with a massive hit for RAID-6 writes, of course) - let alone if you hit the high watermark and are basically stuck at disk speed (terms very between controllers). A good SATA disk is still maxing out at ~100 IOPS per spindle, while 10k FC will be double, and a good SAS disk can almost hit triple that. :)
 
Write latency is always the killer for NAS. With iSCSI you need to consider the IO queue limits. With NFS you need to have a fast response sync mount.

A spinning disk (single stripe say) can do 100 iops on a 64k packet, or about 6MB/s. A write cache will speed this up. Or example a pair of X25-M SSD drives as an slog can do about 2400 iops or 150MB/s.
 
Depending on how fast it spins (literally), yes.

FC can also have IO queue limits, even with cache. Some arrays are notorious for this, others less so. :) (check the VMware KBs on HDS arrays for architectural requirements for that particular series - they're a great example).
 
I'm trying to understand the whole local vs network-based storage for ESXi.

This thread seems to have died down a bit so I hope this won't be seen as hijacking. . . and since this is pretty related, I thought I'd ask a few follow-ups to the original post that really sorta just explores the "theory" behind all this.

First, I'm a network-based storage noob. And pretty much a noob w/ virtualization as well (except at the desktop level). Having said that, I'm looking to deploy ESXi on the cheap in an education environment (we won't lose money if it's down. But if it's down for more than 24 hours, I'll definitely lose some pride!).

I keep seeing conflicting information about whether to store VMs locally or on a NAS-type device. My current plan (and config) is a Raid5 array of 1.8-ish TB on an Adaptec 5405 (3x1TB w/ 1 Hotspare). Please don't recommend more disks or a better controller, we're broke! :D

We'll probably be hosting 6-8 machines on this host. CPU and RAM should be adequate (dual quad Xeons, 12GB). But I keep getting hung up on the possible storage system bottleneck if more than one machine starts needing to read/write a large amount (for example, one user is FTPing a large file onto the web server while someone else is dragging a large file to their desktop from the file server).

I keep hearing that a storage array is the solution for this type of an issue. So I guess my question is: What is it about the storage being in another box across the network that makes it so much better at handling the Disk I/O. Or, am I making a wrong assumption there and those who are recommending a network storage array are also assuming that I'll be able to then use separate disks (or two, perhaps in RAID 1 for protection?) within the array for each VM?

Sorry for the complete noob question. But I'm just having a hard time wrapping my head around the NAS vs Local benefits. And, more to the point, why there are performance benefits since presumably the NAS is running RAIDed disks too.

Sorry that I haven't had time yet to just "RTFM". . . got so many plates spinning (at work and home!). So something concise from the resident experts would be great. Or, failing that, even just a link to the specific "FM" that I need to "R." :p
 
You can make a local storage array that meets most performance requirements, however the big feature gotcha's with VMware NAS/SAN storage is: they are needed for vMotion, HA, DRS and the other 'neat' features that enterprises love.
 
Spindles are spindles for direct to disk performance. That's all that matters. A good array will buy you more with plenty of cache (I don't count anything under 16G as significant, but my exposure is a bit unusual for most people here), but underneath it all is spindles - you have to write the data to disk.

5 SAS disks in a local enclosure are just as fast as those 5 SAS disks in an array (minus cache again) - the array just buys you centralized storage that is shared.

Now, all arrays DO have cache - but most cheap affordable ones don't buy you much except burst control and avoiding seek times for most things - you're still relying on spindles as your chief performance metric. If you're looking at a top-end EVA/CX4/DMX4/9990V or the like, then of course the cache buys you tons, but you're probably not looking in the multi-million dollar deployment range for one host ;)

This is all outside the VM world - basic array fundamentals.

VMs - you create a "container" if you will called a datastore. Multiple VMs live on there just like multiple vms live on your hard drive if you're using Fusion or Workstation or the like. It's just a filesystem, and the VM hard disks are just files.

Oh, and make sure whatever controller you do have - get teh battery backed cache. With write cache disabled, you lose tons of performance since you have to wait for the disks to ack the write before the host can issue another/get the return.
 
You must log in or register to reply here.
Back
Top