• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ZFS: non-optimal RaidZ2 disk-setup

H

hotzen

Limp Gawd
Joined
Jan 29, 2011
Messages
349
Hello,

since the most HBAs have 2*4 SAS/SATA Ports, I also plan to fully utilize them, which means I want to create an 8-disk RaidZ2 vdev.
This means I have 6+2 (N+P) disks, where N=6 ist neither 2^2 nor 2^3 but 2^2.58 which means this configuration is generally not recommended and not considered optimal.

So, is there any experience regarding this setup?
I really dont want to waste 2 ports just because such a setup would be slightly inferior and someone would only see differences in synthetical benchmarks...

Appreciating any advice, thank you.
 
AFAIK, the performance difference is small (a few percent) - I wouldn't let that dissuade you.
 
Many people are using 8 disk raidz2. I am. I have for many years been on the OpenSolaris forum, with Oracle/Sun kernel ZFS engineers answering our questions - and I have never seen anyone mentioning you should have a particular number of disks in a vdev. I just read it here recently, for the first time.

In other words, it should not matter how many disks you use. I am not even convinced you should have a certain nr of disks in a vdev, I have never read it anywhere else. And I am quite knowledgeable of ZFS.
 
The optimal disk configurations are described in detail in Sun/Oracle's ZFS best practices guide, including a description of why. I would assume anyone quite knowledgeable in ZFS would be familiar with this document. See here: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide

I've recently benchmarked almost every possible configuration of 10 disks on my test system. The difference between a non-optimal raidZ2 of 6+2 drives vs an optimal raidz2 of 4+2 or 8+2 drives was small, but not negligible. I am traveling this week and don't have my full study notes on it, but as I recall it was on the order of 10% after normalizing for the performance added by having more spindles in the array (or lost for having fewer in the 4+2 case).

In practice, it only matters for high performance SAN installations. In most cases, the home SAN devices being discussed here are ultimately limited by the performance of their network interface. Since even a 3-drive raidz will normally outperform a singe GigE connection then you can't ever read/write your NAS faster than the disks can perform - even if they are not configured in an optimal way.
 
yep, what they said. go with 8 drives and you'll be very happy with the throughput (likely 4-500MBs+)and never notice the diff. in the real world.
 
Thank you piglover, would appreciate if you posted your benchmark results later, very interesting.
 
About a year ago Sub.Mesa took the initiative to test out performance on zfs with various configs of vdevs and raid-level (raid-0, raidz, raidz2, raidz3 etc), using his ZFSguru installation.

Results with graphs etc can be viewed here: http://hardforum.com/showthread.php?t=1546137.

I plan to expand my zfs-server to a 2nd Norco with 20 Samsung F4s, hopefully before christmas when it should be abit less hectic at work. When I do I'll break in the disks using the benchmark with ZFSguru.

Sub.Mesa also made a post whats the optimal number of drives in a vdev using 4K-disks, in this post: http://hardforum.com/showpost.php?p=1036154326&postcount=2
 
You can always run some benchmarks yourself before putting any data on them. I'd bet that you're 6+2 array will easily out perform a 4+2 array on any large sequential work. The 2 extra spindles will more then make up for the non-alignment penalty. The way I think of it, if you hit the ideal powers of two you get an extra boost but in general it isn't worth worrying about unless you're just a single disk off or designing enterprise arrays.

I've been running a 12 disk raid-z2 for a while without any issues and have been happy with the performance.
 
From memory the optimal number of disk per configuration only came to light when 4kb sector disks surfaced.

If we aren't using 4kb disks is this actually an issue? or do 4kb sector disks just amplify this?

Paul
 
You must log in or register to reply here.
Back
Top