ZFS N00b: raidz3 and larger qty of disks?

[brutalizer]

Aesma, why 19 disks? Is it a "magic" number for raidz3?

 

[Aesma]

Well 128 divides exactly by 16 so it's as magic as other such exact divisions, however the smallest ones don't really make sense because the data drives/parity drives ratios make you lose too much space. The minimum in my opinion is 8 data + 3 parity and even that is steep, 37,5% for parity (and there are still other losses with ZFS, all those checksums aren't free), over that the next magic number is 16+3. Since buying cheap rackable enclosure makes me save over buying 24 bays boxes + PSUs + expanders then 16+3 becomes the sweet spot. Up to 2 vdevs the +3*2 can be housed in the server's chassis, if I add a third vdev I'd probably add another enclosure just for the "parity" drives, using the 4th miniSAS port that I would get anyway with a second PCIe card, maybe by then RAIDZ4 will be a reality and I'd get 3*(16+4) into 16+16+16+4*4 if you know what I mean !

 

[uOpt]

ZFS is a professional filesystem, such considerations are not a priority. If you lose too many drives you lose everything anyway.

As a professional filesystem they should realize that a sync/silver priority value as a property for a ZFS dataset (analogous to the fsck priority in /etc/fstab) would be a useful feature.

 

[staticlag]

As a professional filesystem they should realize that a sync/silver priority value as a property for a ZFS dataset (analogous to the fsck priority in /etc/fstab) would be a useful feature.

if zfs isn't professional, I would hate to imagine what you think of NTFS

 

[Aesma]

As a professional filesystem they should realize that a sync/silver priority value as a property for a ZFS dataset (analogous to the fsck priority in /etc/fstab) would be a useful feature.

Can you explain the benefit ? If you're using the drives in production, whatever is being resilvered, your performance will be affected.

 

[uOpt]

Can you explain the benefit ? If you're using the drives in production, whatever is being resilvered, your performance will be affected.

Resilver more important data first. Of course it increases the overall time to completion. But having files you really don't need resilvered isn't uncommon, just think about caches that can be rebuilt from data with about the same amount of resources as resilvering.

 

[Deleted member 12106]

I would use raidz3 for better safety. And with 20ish disks, I would do two raidz3 vdevs of 11 disks each, which means 22 disks in total.

So if you have an additional 6 disks in the PC case you can have 22 disks in total for two raidz3 vdevs. And I would make sure to use disks like this:
vdev 1:
JBOD case holds 8 disks + 3 in the PC case

vdev 2:
JBOD case holds 8 disks + 3 in the PC case

This way, if you loose your PC somehow, you can still start up the JBOD case, all data are intact.



Compare to this setup:
vdev 1:
-JBOD case holds 11 disks.

vdev 2:
-JBOD case holds 5 disks and 6 disks in the PC case.

In this last setup, if you loose your PC (it gets stolen) you have lost all your ZFS data. Because you can not boot up the JBOD case, because vdev 2 is corrupt as it only contains 5 disks. So, make sure the parity disks are in the PC case (distribute them to the PC case).

So if I do raidz, I would want 9 drives in a vdev,
If doing raidz2, 10 drives per vdev
if doing raidz3, 11 drives per vdev

the "data" are a power of 2

Am I on the right track?