Currently running 8x 4TB drives in single raidz2 vdev, going to add 16 more disks. Random performance is going to suck anyways so I'm going to max out on Seq to saturate 10G.
Some options (?) :
1) 4x 6disk raidz2
If I had to pick now without planning, I'd go for this one because:
6disk per vdev has optimal stripe size.
Future proof; any bigger vdev with 6TB+ disks sounds dangerous.
2) 3x 8disk raidz2
Better than 1) capacity-wise, non-optimal stripe size. No idea about performance vs 1).
3) 2x 12disk raidz3
.. meh.
Mirrors are not an option, don't want to waste that much space.
About optimal and non-optimal stripe sizes: (from freenas forums)
sub.mesa wrote:
As i understand, the performance issues with 4K disks isnt just partition alignment, but also an issue with RAID-Zs variable stripe size.
RAID-Z basically works to spread the 128KiB recordsizie upon on its data disks. That would lead to a formula like:
128KiB / (nr_of_drives parity_drives) = maximum (default) variable stripe size
Lets do some examples:
3-disk RAID-Z = 128KiB / 2 = 64KiB = good
4-disk RAID-Z = 128KiB / 3 = ~43KiB = BAD!
5-disk RAID-Z = 128KiB / 4 = 32KiB = good
9-disk RAID-Z = 128KiB / 8 = 16KiB = good
4-disk RAID-Z2 = 128KiB / 2 = 64KiB = good
5-disk RAID-Z2 = 128KiB / 3 = ~43KiB = BAD!
6-disk RAID-Z2 = 128KiB / 4 = 32KiB = good
10-disk RAID-Z2 = 128KiB / 8 = 16KiB = good
Please, give me your opinions, thanks
Some options (?) :
1) 4x 6disk raidz2
If I had to pick now without planning, I'd go for this one because:
6disk per vdev has optimal stripe size.
Future proof; any bigger vdev with 6TB+ disks sounds dangerous.
2) 3x 8disk raidz2
Better than 1) capacity-wise, non-optimal stripe size. No idea about performance vs 1).
3) 2x 12disk raidz3
.. meh.
Mirrors are not an option, don't want to waste that much space.
About optimal and non-optimal stripe sizes: (from freenas forums)
sub.mesa wrote:
As i understand, the performance issues with 4K disks isnt just partition alignment, but also an issue with RAID-Zs variable stripe size.
RAID-Z basically works to spread the 128KiB recordsizie upon on its data disks. That would lead to a formula like:
128KiB / (nr_of_drives parity_drives) = maximum (default) variable stripe size
Lets do some examples:
3-disk RAID-Z = 128KiB / 2 = 64KiB = good
4-disk RAID-Z = 128KiB / 3 = ~43KiB = BAD!
5-disk RAID-Z = 128KiB / 4 = 32KiB = good
9-disk RAID-Z = 128KiB / 8 = 16KiB = good
4-disk RAID-Z2 = 128KiB / 2 = 64KiB = good
5-disk RAID-Z2 = 128KiB / 3 = ~43KiB = BAD!
6-disk RAID-Z2 = 128KiB / 4 = 32KiB = good
10-disk RAID-Z2 = 128KiB / 8 = 16KiB = good
Please, give me your opinions, thanks