New ESXI server - RAID 10 disk performance (benchmarks included)

MassiveOverkill said:
So once again I'll agree that we agree.

Lopoetve discounts the whole premise of partition alignment, unless you're using Netapp. He then slightly backs off this statement stating it can have a noticeable impact in other apps. You say you agree with, but then state your own peeps back up what I'm saying, for properly deploying VMs PA'd.

I'm not trying to discount Lopoetve's VM knowledge. He's obviously seasoned. I'm sticking with my guns on this issue and I'll be happy to take the discussion directly to VM's own forums to discuss it. I'll gladly recant when VMware and Microsoft publish something that says mis-aligned volumes no longer affect disk performance. I'm talking about the misaligned volume itself, not the fact that all of MS latest OS offerings now properly partition-align.
Click to expand...

No, I don't discount it. Average performance increase on most arrays is less than 5%. For the gross majority of customers, this is considered an insignificant number - by the time they're pushed past the performance threshold and have finally over loaded their array, they're SO far past the 5% mark that it doesn't matter - alignment would not have saved them. For the rest of the people not blowing up their arrays, RAID type, spindle speed, spindle count, etc are most important.

We simply leave it to the hardware vendor at this point, as they may have specific recommendations - all but netapp, to this point, have pretty much come out with "Yeah, sure, if you really feel like it, but it doesn't get you a lot on our array." NetApp performance on an unaligned WAFL filesystem with the default 4k allocation size is horrendous, so they not only make recommendations, but provide a very easy to use tool to do so as well.

At this point, alignment is considered such a small portion of the performance pie that we do not include it on our storage performance customer presentations anymore even, or vmug presentations. Again, with the exception of netapp.

Will it hurt? No. Will it buy you hidden unlocked powers of massive performance? Not anymore. AT least, not with anything that has been seen out there.
 
Last edited:
MassiveOverkill said:
Fair-enough, but there's a good reason why there was no response to my rhetorical question:

"You're saying Microsoft has remained silent on partition-alignment on it's OS flavors?"

because this is heavily documented.
Click to expand...

Yes, but what storage vendor recommends it or provides guidelines? The one doc out there was written by EMC, and has been effectively withdrawn (in fact, I'm surprised it's still out there). MS may scream "yes, align us!" but the storage vendors are, as a whole, pretty much just going "meh".
 
Alright - replaced the bad disk and WOW - what an improvement!


aaYZV.png

ZEd9e.png

pH7Ov.png
 
lopoetve said:
Our recent performance testing shows that it isn't really required anymore.

You tell me what SQL test to run, and I'll run it on a variety of arrays, and send the results here, and we'll let them speak for themselves (IN a VMware VM).
Click to expand...

I can send you the full script I made using SQLIO, but if you want to do it yourself, I'm mainly looking at 8K Random Writes using:

SQLIO -kW -s%TESTLENGTH% -frandom -o8 -b8 -LS -F"%WD%\PARAM.TXT" >> "%WD%\SQLIORESULTS%COMPUTERNAME%\RANDOMWRITE8K.TXT"

where:

%TESTLENGTH%=300
%WD%=%CD%

Contents of PARAM.TXT:

%DRIVECHOICE%:\SQLIOTESTFILE%COMPUTERNAME%\SQLIO_TEST.DAT 4 0X0 %TESTSIZE%

Where %DRIVECHOICE%=drive letter you're testing
and %TESTSIZE%=3000

This was done on a rather simple Dell R710 with an E5520 with 6 gigs of RAM.

Disk sets were 2 disk in RAID1 for OS and 4 disk 7200 RPM SATA in RAID 10 for data using Dell PERC6. I believe the RAID stripe sizes were default for setting up PERC6 (if memory serves 64K) The RAID10 data array was tested using the above SQLIO test.

It would be nice if the VM Guest test was ran on a simiilar array, meaning only 4 disks........RAID 10 would be nice, but RAID 5 will do, and they don't have to be SATA, if SAS is the lowest disks you have, that's fine, even if they're 15K, so long as the test is done with the VM guest disk partitioned with 63K offset and then the next test done with 1024K offset. Both formatted 64K File Allocation Unit Size.
 
I can probably make that test happen. Let me see how many free disks I have in the lab on my CLARiiON.
 
Sorry, been out sick as a dog for the last week (or most of it). I'm building VMs now.
 
I'll be running these on Netapp and clariion luns too - not local disk.
 
There's no doubt that it's a good idea, it'll never hurt - it just won't help as much as it used to either. For most arrays.
 
Ok, I'm still testing, but we've got some interesting results so far. I won't spill the beans yet, but suffice to say 2/3 of the arrays so far act as I expected - 2-4% difference under extremely heavy load at most. But there was one that matched up with the crazy improvement (almost 100% more performance in IOPS, and 50% less latency) - we're double checking the numbers against another one of that array and testing a few more.
 
well, I'm waiting for a replacement SP for the Clariion (yay cache module being dead), and waiting for the Equallogic, NS20, and NetApp iSCSI volumes. So far, the NetApp FC volume, LeftHand VSA, and OF NFS storage all show almost identical performance aligned or unaligned using the testing methods I have (set of iometer tests). I'll dig more once the SP comes in.
 
SQLio is not part of our testing suite, but I'll give it a try at some point. I don't expect significant differences given what tests I'm currently running though.
 
I have to admit, this thread has been nothing but entertaining. My issue, is if one guest is only seeing 2-4% improvement, then I'd agree why bother. The problem isn't that simple depending on the situation, and I think lopoetve and fancy pants VCDX NetJunkie would agree. For instance, my previous job we had an EVA4400. Not a bad array, but it is only a mid level array, so if you put enough 20-40 GB VMs on that thing you can push it to the brink. I won't even bother getting into ALUA and the hell I went through with ESX 3.5.. thank god for 4.0 and its ALUA awareness. Storage people didn't understand what "not multipathing aware" means sometimes, I swear. Anyway, that entire environment was almost all p2vs. There were fresh VMs, but mostly p2vs, so that means no alignment on those (by default). Most of those p2vs were done before I got there, I aligned all the ones I did as part of the process. No mbralign either, I had to use gparted :(. I cannot tell you the amount of hours I spent ... but anyway.

The bigger point, is that if you had a nearly saturated storage system caused by a high number of VMs (rather than a fewer number of high IOPS consuming VMs), and there is something causeing extra I/O to be performed per VM (like partitions not being aligned), it can make a big difference in the big picture.

Seperately, but another example of not looking at the bigger picture, I'd say we also saw quite a large drop in CPU utilization from switching every Windows server to PVSCSI across the board along with VMXNET3. Linux VMs we changed the data volumes to PVSCSI. The gains per VM might not be THAT much, but the benefits of paravirtualization moving processing out of the virtualization to be performed by the host is significant. Big picture view can help here, no doubt. I wish I had more concrete numbers on the drop in CPU resources, but it was very apparent to me.

BTW, if you hadn't noticed, PVSCSI is supported for booting Windows, not Linux. Previously neither could boot from PVSCSI, at least not "supported". PVSCSI can be used with VMs with less than 2000 I/Os as of 4.1 (could lower performance in earlier releases) see: http://kb.vmware.com/kb/1017652 you'll have to use your magnifying glass to see the sentence saying that 4.1 corrected the issue. PVSCSI and VMXNET3 can BOTH be used with Fault Tolerance as of 4.1. Good luck finding all that documentation in one place. Its not easy, being cutting edge ;). Either way, USE THEM. BOTH. Kthx have a nice day!
 
zpackrat said:
If you allocate the VMFS from withing the console after installation the disks will be aligned.
Click to expand...

If you're deploying Server2003, WinXP, or any older guest OS, the data disks within the guest still have to be manually-aligned unless you're using a template that has already had them aligned.
 
NetJunkie said:
<fancypants> Well there's your problem! :p </fancypants>
Click to expand...

YOU ARE A FANCY PANTS. I hope to be one eventually ;).

Yeah, no kidding. The EVA has its place, but trying to run a 150+ VMs on there, some with high I/O requirements was a PITA. We did get an EVA4000 to put T&D on, so that helped a little. The other issue, was they were too cheap to buy disks for backups, so the temporary mount point for VCB at the time was ALSO on the array. VMs were so sluggish at night during backups, it could make you cry if you had real work to do.

The first problem when talking to other IT people at that place: What are IOPS? :: FACE PALM :: Sometimes it is just not possible to spend money the best way because of politics, and it results in lots of reactive spending that actually costs more in the long run -- just not during THAT budget year. Lame-o.

Oh well I don't work there anymore (thank you lord). The VMware environment here is big as in HUGE, and um, they spend money like they can print it. Say what?
 
Last edited:
You must log in or register to reply here.
Back
Top