• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

5 or 6 drive ZFS? ZIL?

M

m0po

n00b
Joined
May 14, 2014
Messages
17
I currently have a four drive Synology DS413 with 4x3tb drives, however I am running very low on space. I am also looking for more flexibility so I am looking at ZFS.

Questions:
- Can I make a 6 drive RAIDZ?
- Do I need a SSD for SLOG/ZIL/whatever it's called?
 
You can, but you will be wasting space. The math is found here on this board or elsewhere in the webs.
 
m0po said:
Questions:
- Can I make a 6 drive RAIDZ?
- Do I need a SSD for SLOG/ZIL/whatever it's called?
Click to expand...

6 disks per Raid-Z is bad (especially with high capacity disks where a rebuild can last a day)
The probability of a total data loss is too high.
I would prefer a 6 disk Raid-Z2 where any two disks are allowed to fail without a dataloss.

A ZIL is a LOG device that is used for applications that request secure sync writes
where every single write must be confirmed from disk until the next one can occure.

This is true with ESXi and NFS datastores or databases. A usual SMB filer does not request sync.
So the need of a dedicated ZIL depends on use case. If you do not add a dedicated ZIL, you use
the pool as ZIL. The advantage of a dedicated ZIL is performance with sync writes.

On a slow pool, the sync write performance can go down to 10% of the write performance in
regular (nonsync) mode where you use the RAM as ZFS write cache.
 
Last edited:
_Gea said:
6 disks per Raid-Z is bad (especially with high capacity disks where a rebuild can last a day)
The probability of a total data loss is too high.
Click to expand...
This is a myth which needs to die. Mean time to data loss with 6x3TB drives in RAID-Z is over two thousand years. Also, it wouldn't be a total data loss unless the user didn't back up their data. Surely nobody would be THAT foolish in this day and age.

Granted, it's possible that uptime is incredibly important to this user, and a 0.4% chance of losing said uptime in the next ten years is too big a risk for them to take.

Source
 
Last edited:
With consumer level 3 or 4tb sata drives, typically all purchased at the same time from the same batch? No, I won't trust a raid-z1 with that math.
 
I have had a double disk failure in a Raid-5 and I am not 2k years old.

I expect annual failure rates (depends on model and age) of single disks between 2% and over 10% per year. In a Raid you must multiplicate this with number of disks. If conditions like temperature are not the best or disks are older than a few years this can be even higher.

In my own setups I use about 200 disks of different age with about one failure per month. Single disk failures is something that happens. The danger of a second failure during rebuild is not acceptable for me. I do backups but backups are always from yesterday.

This experience is similar to
http://www.tuaw.com/2014/01/31/backblaze-stats-on-27-000-hard-drives-show-which-ones-keep-on-ti/

I would not use or suggest Raid-5 or Raid-Z1 with many disks or high capacity!
 
dave99 said:
With consumer level 3 or 4tb sata drives, typically all purchased at the same time from the same batch? No, I won't trust a raid-z1 with that math.
Click to expand...
The math assumes some very pessimistic values. Bring them more in line with real world estimates, and you end up with an even smaller chance of losing data. Trust it or don't... Your choice. Just please don't spread the myth around. This community doesn't need the misinformation.

_Gea said:
I have had a double disk failure in a Raid-5 and I am not 2k years old.
Click to expand...
There will always be statistical outliers, thus the "mean" part of "mean time to data loss".

_Gea said:
I expect annual failure rates (depends on model and age) of single disks between 2% and over 10% per year. In a Raid you must multiplicate this with number of disks. If conditions like temperature are not the best or disks are older than a few years this can be even higher.
Click to expand...
Hot damn! I'm GUARANTEED to have a disk fail if I run 10 disks? That's crazy! (I know, I know... Just poking fun at ya! :D)

The danger of a second failure during rebuild is not acceptable for me.
Click to expand...
This is really what it comes down to - Is the 0.4% chance of needing to restore from backups acceptable or not? Stats are funny like that, though. A 0.4% chance of losing two disks in one array could expand to a 100% chance of losing 2 disks in a raid-5 array if you have a thousand arrays. For a home user, with a single array, though? The chance of losing two disks is so small as to be insignificant.
 
Last edited:
I'd guess that MBTF numbers are based on average conditions (probably lots of relatively idle performance with occasional read/write), I'd argue a raid rebuild is not average condition, it's working all drives hard for potentially 24 hours or more non-stop.
 
I expect annual failure rates (depends on model and age) of single disks between 2% and over 10% per year.
Click to expand...

I expect the same. And also these numbers were about the same for Google and BlackBlaze (well except for the 1.5TB Seagate Greens that BlackBlaze found unusable in their environment).
 
_Gea said:
MTBF of many disks indicate that a failure is expected every 100 years or less
Click to expand...
MTBF is worthless for a consumer. It doesn't mean what you think it means. This is the simplest link I could find to explain why. The dailytech link is even a bit of a joke - The original paper doesn't mention MTBF at all. It uses MTTF, which is a VERY different metric, and even that is not well understood by most folks.

edit: Imagine you see a drive advertised with a MTBF of 1 million hours. This means that if you had 1 million drives running in perfect conditions, you could expect one to fail per hour. MTBF is calculated by a lab over a period of a few weeks. For example, they'd take 100 disks, start them up, and let them run until one failed. Then they'd note how long it took for that drive to fail, add in the number of hours the other 99 drives ran without failing, and then work some magic math to come up with the MTBF. The other 99 drives could have been ready to fail only an hour later, but it wouldn't be reflected in the MTBF number.

In short, MTBF is not intended to indicate average drive life.
 
Last edited:
hmm.. so if i do a 6 disk raidz2 with 4tb disks, what kind of read speeds can i expect?150mb/s? 200mb/s? assuming they are hitachi coolspins and an asrock E3C226D2I motherboard with 16gb ram.
 
TeeJayHoward said:
The math assumes some very pessimistic values. Bring them more in line with real world estimates, and you end up with an even smaller chance of losing data. Trust it or don't... Your choice. Just please don't spread the myth around. This community doesn't need the misinformation.
Click to expand...
Actually, you are wrong. That is only a MODEL, and as we all know, models are a simplification of reality. Dont trust that model, has it been proved correct by research community, or is the model something a random guy tossed up? I can create a model too, that shows your model is wrong. Which model will you trust then? My model, or a model that the research community relies on, or statistics from real life? Models and theory is one thing, but if real life examples says something else, I draw the conclusion that real life is correct.

It is like when hard disk vendors specify that a SAS Enterprise hard disk has 1 irrecoverable error for every 10^16 bits read. Do you draw the conclusion your disk is safe? You shouldnt, because there are lot of external error sources, for instance the SATA cable might be slightly loose, the power supply flaky, etc etc etc etc. This results in a much worse number than 1 error in every 10^16 bits read. For instance, NetApp researchers showed that 5-10% of all data corruption was because of buggy disk firmware, and they examined many disks in that study (potentially millions). In NetApp case it does not matter how safe the disk is, according to vendors models, there are other external factors that make data corruption a lot more common than "1 irrecoverable error for every 10^16 bits read".

Thus, to trust a model that some random guy cooked up - is not really optimal. Fact is that sysadmins see disk crashes all the time, that is the reason raid-6 was invented. Not because of some model predicting double disk crash is rare, but from reality examples the need for raid-6 arose. Talk with Backblaze or Google and see if they rarely encounter double disk crashes or not. You will be surprised.

Here is more information on double disk crashes and why raid-5 is not a good idea with large disks. In this article there are more links with good information.
http://www.zdnet.com/blog/storage/why-raid-6-stops-working-in-2019/805
 
TeeJayHoward said:
MTBF is worthless for a consumer. It doesn't mean what you think it means.
Click to expand...

ok, if MTBF is worthless (and I'd agree with that), then why you are using it as a source in your post higher up? That raid loss calculator you posted uses mtbf as it's main data source.

I'm not trying to troll, I'm trying to understand your point. You say raidz1 is super duper safe because these statistics say so and this myth that it's not needs to die. Yet there are people getting burnt by single parity raid failures all the damn time. I'd say that myth (if it is), needs to be be reinforced that raid5/raidz1 isn't good enough if you value your data at all.
 
If raid-5 was a super safe, there would be no need for raid-6. Now they are talking about the need for raid-7 (or what they call it, when three disks are allowed to crash). Why are people demanding raid-6 och raid-7 if raid-5 is super safe? We have (at least) two choices:

1) Raid-5 is not safe, double disk crashes happen. Especially when you repair a raid with big disks (6TB) which takes several days (weeks?) - during that heavy stressful lengthy repair process another disk might show read errors - which means you have lost your entire raid-5. Yes, if another disk shows read errors - you are toast. Therefore you need at least raid-6 with large disks. And with 10-12TB disks it might take 1-2 weeks, so your raid is vulnerable during repair process (which stresses the disks a lot). And disk sheets specify "1 irrecoverable error on 10^16 bits read" - and if you do the math, it will show that there is 1 in 10 or so - that you get a read error when repairing a raid with 5TB data. So if you repair your raid-5 ten times, one of the repairs will show read errors - which means you have lost all data. I dont remember the exact numbers, but it was something like that. Or maybe even worse. Read my link above more information on this.

2) People are wrong. All sysadmins that report of double disks failing (one crash, one read error) are fake. This is just FUD.

Now, what is most likely true?
 
Especially when you repair a raid with big disks (6TB) which takes several days (weeks?)
Click to expand...

If it is that slow I believe you have way too many disks in the array, too much disk activity while rebuilding, bad settings or a very slow raid implementation possibly caused by too many expanders. I mean it takes 9 to 9.5 hours for me to replace a 2TB 7200 RPM drive in a linux software raid6 array with 10 to 12 total disks even on a 6 year old system with a core2quad cpu.

Edit:
Although I expect 6TB disks will be close to 30 hours I do not have any yet..

So if you repair your raid-5 ten times, one of the repairs will show read errors - which means you have lost all data.
Click to expand...
Having a single sector corrupted in an entire raid array does not mean you have lost the entire array however if it is metadata it may mean more than 1 file is corrupt.

Now, what is most likely true?
Click to expand...

At work I have moved from raid5 4 or so years ago. What changed my mind is I had a double disk failure on a 10 disk raid5 array. With some work I was able to recover by using ddrescue to image the second disk kicked out of the raid array and linux software raid allowed me to force the array to reconstruct with the disks I tell it to use in the order I tell it. After this I avoid raid5 for the most part.
 
Last edited:
TeeJayHoward said:
The math assumes some very pessimistic values. Bring them more in line with real world estimates, and you end up with an even smaller chance of losing data. Trust it or don't... Your choice. Just please don't spread the myth around. This community doesn't need the misinformation.

There will always be statistical outliers, thus the "mean" part of "mean time to data loss".

Hot damn! I'm GUARANTEED to have a disk fail if I run 10 disks? That's crazy! (I know, I know... Just poking fun at ya! :D)

This is really what it comes down to - Is the 0.4% chance of needing to restore from backups acceptable or not? Stats are funny like that, though. A 0.4% chance of losing two disks in one array could expand to a 100% chance of losing 2 disks in a raid-5 array if you have a thousand arrays. For a home user, with a single array, though? The chance of losing two disks is so small as to be insignificant.
Click to expand...

The home user has probably a weaker backup strategy, typically no access to a tape robot, weaker protection against power outages/surges, weaker temperature and humidity control, weaker protection against interference from dust, animal hair, children messing with the server. Also housing backups at a separate address might not be easy, with my current DSL connection I have no hope of putting my backup server elsewhere for example.

So it's RAIDZ3 for me.
 
You must log in or register to reply here.
Back
Top