Please check my logic on storage solution decisions

[Konstantinnovation]

I read many things about ZFS implementations vs traditional software/hardware RAID setups. That being said I wanted to know if anyone has any opinions on the following:

1. Use ZFS with mirrored vdevs.
I did a lot of research but wanted to confirm that this was not leading me into my doom:
http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/
It basically argues that I should not look into RAID6 for larger drives, and should instead use zfs to stripe across 2 disk mirrors.

2. Implementation and upgradability:
I would use 4x 4TB hdds now, setup ZFS with two 2-drive mirrors.
At a later date I would add 3 more drives and use one as a hot spare... and upgrade as needed after that.

3. Hardware: (I am not 100% certain about this part)
I was planning on going with WD RED NAS 4TB drives ( WD40EFRX )

I value your input
-K

 

[mwroobel]

Before I comment further, can you answer some initial questions? What is the use case for the array(s) you are building (Write heavy/Read heavy workloads?) NAS/SAN/DAS? Preferred operating system(s) for both clients and server(s)? Network type? Backup Plans? You mentioned 16TB of drives initially but what do you forsee your expectations for expansion over the next 3+ years? What enclosure are you planning on using (future expansion may be limited for example if you get too few slots or too small drives?)

 

[thedge]

1 - Personally... I prefer RAIDZ2 over mirrors unless you are talking about a VM data store. Mirrors for plain simple file storage make no sense, you take the hit of efficiency (Dont get greedy, 50% is fine? WTF does he do for a living? Give him a 50% income tax and lets see....) and you do not get "lose no sleep" redundancy at least in my mind. In mirrors yes you can lose 50% of your drives and no data, but only if you lose the "right" 50% of your drives. If one drive dies and a second follows, you have to hope its not the second in the same mirror pair. As someone who has had two drives start dying at the same time more than once (Seagate!), I do not like leaving it up to chance that the second dying drive is not the mirrored pair. But that is me. If you are doing something involving huge amounts of sequential writes, etc then mirrors start making more sense. For someone at home with home type data, mirrors do not make sense to me except for homelab VMs.

2 - Again, IMHO... Buy the two extra drives now, RAIDZ2 now, relax.

3 - I have 8x 2TB Reds running in my NAS in a RAIDZ2. Not a single issue at all, has been running for probably 18 months non stop. Maybe more.


If youre using it as a VM datastore, skip the spinning disks and get SSDs in whatever RAID level you want including mirrored. I have four WD 300GB Raptors in mirrored pairs only for VM datastore and will eventually be replacing them with SSDs because SSDs just whip the pants off spinning drives for VMs. I just wish the Raptors werent so reliable so I could replace them sooner.

The link has the backup part right. Backup. Backup. Backup.

 

[Konstantinnovation]

mwroobel
Use case: Media Server primary storage / Network storage to back up: OS images, documents etc.
Mostly sequential reads, with weekly backup writes + occasional updates of moderate sized files.

Server OS: Currently going with Ubuntu, unless I have a compelling reason to change.

Client OS: Windows7, windows10,Linux(mint/ubuntu)

Network: gigabit switched, only laptops and tablets are wireless. Will be wiring the house with ethernet as the next project, but that won't change much.

Primary Backup: I have 10TB worth of external drives, though some of them are mirrored now.

Secondary Backup: Dropbox + btsync to some VM's elsewhere for absolutely critical (if my house burned down)

Expansion: Uncertain, though more is better. I have 4TB of unique data at the moment, though not all of it is critical. I was aiming at the 12TB to last me at least 3-4 years. Beyond that I don't know what would be required.

Enclosure: Full tower with far more drive space than I would ever wish to fill. I have just started looking into the ECC thing, and evaluating if i should make this a NAS, or part of the server. It would be much cheaper to make it a cheap i3 NAS instead of replacing the server's processor with something on par with the i7 (and multi threading is important for the non-disk operations).

thedge:
-I am very happy to entertain the option of RAIDZ2, and it is safe to add a hot spare later!
-The logic behind the single drive redundancy, is that the rebuild is so fast and read-only to the old drive,s so it was worth the risk. It is debatable as to which is safer especially with a hot swap for the mirrors. Truly simultaneous failure is not very likely. Usually the failures happen
-I don't really deal with VM's, though that is on my to-do list.


With raidZ2, more drives are better ratios, so the question might be:
A- 6x 4TB -> 16TB usable - lower fault rate (due to fewer drives)
or
B- 8x 3TB -> 18TB usable - higher fault rate (more drives)