• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Windows 2012 SAN

C

Child of Wonder

2[H]4U
Joined
May 22, 2006
Messages
3,270
Going to build a Windows 2012 SAN to see how it performs compared to ZFS. Here's the specs:

AMD Athlon II X2 2.8GHz
32GB RAM
8x 2TB 7200RPM drives
2x 240GB Intel 330 SSD drives
Intel PRO 1000/VT quad port Gb NIC

I plan on creating a Storage Pool comprised of the 8x 2TB drives with one as a hot spare. Then I'll create a mirrored volume across the 2x 240GB SSD drives. Using a program called FancyCache, I can use the RAM as L1 read cache and SSD as L2 read and write cache for the SATA drives.

On the 2TB drives there will be a 6TB parity volume with no cache in front of it (CIFS share) and then some smaller parity volumes that will have cache enabled and will be used for virtual machine storage via iSCSI and NFS for VMware and CIFS for Hyper-V 2012.

I'll post results once it's built.
 
I just built one using 2012 Storage Server, interested on seeing how this turns out using storage pools. I haven't delved in that deep just yet, but I figured 2012 gives me the option of running vCenter on it at the same time, so that's one less box I have to worry about as well.
 
First results:

ATTO Benchmarks
8x 2TB 7200RPM Storage Pool

Simple Virtual Disk: 905MB/s read, 890MB/s write
Mirror Virtual Disk: 467MB/s read, 450MB/s write
Parity Virtual Disk: 908MB/s read, 38MB/s write

Yes, you read that correctly. The parity volume maxed out at 38MB/s writes. Apparently Storage Spaces uses journaling in parity similar to ZFS. Without dedicated journaling drives the journal is stored on the Pool itself so you're getting a massive write penalty.

This may be OK for my file share volume but not for my virtual machine volumes.

Next up I'll be testing FancyCache in front of the drives as read/write cache.
 
I have never really been a fan of using apps like FancyCache... I mean by default any modern OS uses all of the available ram for cache...
 
Good info to know, i am just building a new storage server and hadnt yet decided on how to put it into use, ZFS, windows variant...

how many drives would you need for dedicated journaling ?
 
Firebug24k said:
Is there an option to move the journal to a different device?
Click to expand...

MrGuvernment said:
Good info to know, i am just building a new storage server and hadnt yet decided on how to put it into use, ZFS, windows variant...

how many drives would you need for dedicated journaling ?
Click to expand...

You can use Powershell to dedicate drives for journaling. 2 SSDs are recommended.

First find your drive's name to use for journaling.

Get-PhysicalDisk

Then add it to the Storage Pool as a journal.

Add-PhysicalDisk -StoragePoolFriendlyName "<your storage pool name>" -PhysicalDisks <physical disk name> -Usage Journal

What I'm reading from others is that even with the SSDs journaling my 38MB/s may go to 200MB/s. Still not great for 8 drives.

However, with FancyCache in front of a parity volume using 8GB of RAM as cache and 100GB of SSD as cache I get numbers all over the place with 512 blocks getting 3.6GB/s (hello cache!) and 8192 blocks calming down to 1.1GB/s read and 660MB/s write.
 
Found this Powershell command:

Set-StoragePool -FriendlyName <Storage Pool Name> -IsPowerProtected $True

With this set to true a parity virtual disk gets 835MB/s reads and 138MB/s writes. If I set it to false and run the test again I get 780MB/s and 40MB/s.

Since my server is plugged into a UPS good for 20 minutes of battery life and all other servers accessing the disk are not power protected, it's probably safe in my case to leave this set to true.
 
Even at 138MB/s write, that's terrible for that hardware setup, I would think. Maybe it's CPU bound? What's your processor utilization for writes?
 
Firebug24k said:
Even at 138MB/s write, that's terrible for that hardware setup, I would think. Maybe it's CPU bound? What's your processor utilization for writes?
Click to expand...

Not bad. Around 25% at worst during the test on the parity drive.

I agree, even 138MB/s is pretty terrible.
 
Beats me, then. I am hopeful there's something we're missing - MS isn't going to sell very many people on it if the performance is that bad.
 
For Sure,

I may have my build parts coming in next week, this week if i am lucky! and i would love to play around with this!
 
When I played around with it in the preview builds, write performance was awful. Microsoft has done nothing to address the complaints about it. It's really not ready for primetime.
 
McTurkey said:
When I played around with it in the preview builds, write performance was awful. Microsoft has done nothing to address the complaints about it. It's really not ready for primetime.
Click to expand...

With the addition of FancyCache I'm really pleased with the performance I'm getting so far. Right now there is a 6TB parity virtual disk with my file shares on it and 2x 700GB mirror virtual disks for my VMs. Performance is good on the VMs, which I'm sure is partly because they're on mirrored vDisks with 44GB of cache in front of them.

Still, I like the ease of management with Storage Spaces but they really do need to improve the performance.
 
All of the testing I've seen indicates that storage spaces is severely lacking in performance.
 
drgnfang said:
All of the testing I've seen indicates that storage spaces is severely lacking in performance.
Click to expand...

Yes, parity virtual disks experience terrible writes.

MS needs to add some intelligence to mirror vdisks so it reads from each mirror as ZFS does and eliminate the massive parity write penalty.
 
Are you guys getting terrible performance on the storage server while data is being written to the storage spaces (any time of virtual disk)?

If I write a couple of hundred GB of data to any volume (mirror or simple, I gave up on parity) the server manager GUI becomes completely unresponsive until 10-30 seconds after the transfer has been paused, completed or cancelled.
 
Why MS can you not learn from others!

All this talk of crap writes is turning me off real fast!

My build in a Nocro 4224 case

 
MrGuvernment said:
Why MS can you not learn from others!

All this talk of crap writes is turning me off real fast!

My build in a Nocro 4224 case

Click to expand...


You should definitely use ZFS on that hardware if it suits your use-case.
 
extide said:
You should definitely use ZFS on that hardware if it suits your use-case.
Click to expand...

Why would he use ZFS, he has a real RAID card? Put that 9280 to good use, I'm using a 9265 with my 2012 NAS, I had thought about using storage spaces, but there's no good reason to with the performance it's showing, if you have the hardware, use it.
 
Mine is purring along just fine. The only virtual disk I have in parity mode is a 6TB file share so the 138MB/s limit on writes doesn't affect me there. My VM datastores are all mirrored and perform great.
 
That is what i wasnt sure about originally, the last time i tried ZFS, well opensolaris variant, i could not for the life of me get it to properly work with my domain, and i didn't want to have to assign a 2nd user account for people.

the box will be the 2nd massive storage box for our office, which will have a 1:1 copy of everything from the main storage box which gets everything dumped to it that is important.

Windows would be good, but this thing will be getting a few hundred gigs written to it daily...
 
ND40oz said:
Why would he use ZFS, he has a real RAID card? Put that 9280 to good use, I'm using a 9265 with my 2012 NAS, I had thought about using storage spaces, but there's no good reason to with the performance it's showing, if you have the hardware, use it.
Click to expand...

The main one is that of performance and maturity. Solaris, Linux, etc perform better than Windows more often than not and their tools have been around for years. MS is just now getting around to adding in features that are anything but new. No matter what OS someone picks the HBA will be needed for expand-ability purposes. You're not chucking it. You are just moving where the RAID functions are performed.
 
kac77 said:
The main one is that of performance and maturity. Solaris, Linux, etc perform better than Windows more often than not and their tools have been around for years. MS is just now getting around to adding in features that are anything but new. No matter what OS someone picks the HBA will be needed for expand-ability purposes. You're not chucking it. You are just moving where the RAID functions are performed.
Click to expand...

Yes, you are chucking it, there's no reason to use a 9280 if you're not going to use the features of the card, use a 9207 instead.
 
ND40oz said:
Yes, you are chucking it, there's no reason to use a 9280 if you're not going to use the features of the card, use a 9207 instead.
Click to expand...

Not really. If I put all of the disks in JBOD mode have I chucked the card? Nope. Could I change my mind and use RAID later? Yup. Of course someone could buy a different card instead. However, not everyone has a time machine. Therefore, LSI has thoughtfully left an option open for those people who need IT firmware but don't own a DeLorean.
 
kac77 said:
Not really. If I put all of the disks in JBOD mode have I chucked the card? Nope. Could I change my mind and use RAID later? Yup. Of course someone could buy a different card instead. However, not everyone has a time machine. Therefore, LSI has thoughtfully left an option open for those people who need IT firmware but don't own a DeLorean.
Click to expand...

You're chucking the features of the card, sure you can use the IT firmware and bypass everything, but then you've just wasted the features of the card. If you're perfectly fine with doing that, then go for it, forget your DMC-12 and the extra money you spent to get the features of that particular card.
 
ND40oz said:
You're chucking the features of the card, sure you can use the IT firmware and bypass everything, but then you've just wasted the features of the card. If you're perfectly fine with doing that, then go for it, forget your DMC-12 and the extra money you spent to get the features of that particular card.
Click to expand...

If you're that price conscious maybe you shouldn't be using these cards.
 
kac77 said:
If you're that price conscious maybe you shouldn't be using these cards.
Click to expand...

It has nothing to do with being price conscious, why buy a card for its features if you're not going to use the features. It's as simple as using the proper tools for the job.
 
kac77 said:
Not really. If I put all of the disks in JBOD mode have I chucked the card? Nope. Could I change my mind and use RAID later? Yup. Of course someone could buy a different card instead. However, not everyone has a time machine. Therefore, LSI has thoughtfully left an option open for those people who need IT firmware but don't own a DeLorean.
Click to expand...

Don't put the drives in virtual disks if using ZFS. This will write metadata to the drives and they will only then be able to be seen by an LSI RAID controller. Make sure to use an IT firmware. It will also use a different driver in IT mode...
 
msitpro said:
Don't put the drives in virtual disks if using ZFS. This will write metadata to the drives and they will only then be able to be seen by an LSI RAID controller. Make sure to use an IT firmware. It will also use a different driver in IT mode...
Click to expand...

Um it was an example...LOL
 
So a few patch tuesdays have come and gone, have those write speeds seen any love?
 
spazoid said:
Are you guys getting terrible performance on the storage server while data is being written to the storage spaces (any time of virtual disk)?

If I write a couple of hundred GB of data to any volume (mirror or simple, I gave up on parity) the server manager GUI becomes completely unresponsive until 10-30 seconds after the transfer has been paused, completed or cancelled.
Click to expand...

I haven't run into that. When I first set the server up I moved about 3TB of data and routinely move/add a few to a few dozen gig to my storage spaces. I have one parity volume and one simple volume.

The only thing I've noticed is that the server manager will run a little slower. Takes a few seconds to refresh vs the one or so otherwise.
 
As far as I know there is no IT firmware for the SAS2108/2208 cards. Also did someone actually check if JBOD mode works with these cards? The last KB article I read suggests that only SAS2208 cards had that option. Of course there is the single disk RAID0 option, but that is not so straightforward like just attaching an additional drive.
 
Reading through the thread, I noticed the Battery backup True/False command, and the Separate Journal location command. Did you ever test the commands in combination?

From your posts, is looks like your final config was with the Battery backedup switch enabled and FancyCache running, correct?
 
Nate7311 said:
Reading through the thread, I noticed the Battery backup True/False command, and the Separate Journal location command. Did you ever test the commands in combination?

From your posts, is looks like your final config was with the Battery backedup switch enabled and FancyCache running, correct?
Click to expand...

Yes but I have not tried dedicated journals. Based on my research no one has gotten more than about 200MB/s using them.

Plus, Microsoft itself says that parity vdisks should only be used for read heavy applications. They're essentially saying "we give up" when it comes to improving parity write performance for now.

Hopefully in a future service pack or the next Windows server release we'll see better parity write performance. In the meantime, if you want decent write speeds mirror is the answer.

Personally, I'm sticking with Windows 2012 and Storage Spaces just because I don't want to fuck around with Linux, Solaris, etc. any more and I want the SMB 3 capability of WS2012.
 
You must log in or register to reply here.
Back
Top