Best drives for RAIDZ (TLER, URE, cost, etc.)

T

the_b_man

Limp Gawd
Joined
Sep 5, 2007
Messages
185
I'm trying to decide on drives, and have been doing a lot of Googling. Unfortunately, no drive seems to meet all the recommendations people are making. Specifically, don't use drives with TLER, and insist on 10^15 URE.

Option 1: WD Red drives
Pros: Cheap
Cons: TLER isn't recommended for ZFS (not needed for ZFS, but some say it could actually be detrimental), URE rate of 1:10^14, which is considered low/risky

Option 2: WD Se (low end enterprise) drives
Pros: URE rate of 10^15
Cons: Expensive, still has TLER

Option 3: Desktop drives (ie. WD Black, Hitachi, Seagate)
Pros: Cheap, no TLER
Cons: URE rate of 10^14

- Will RAIDZ2 protect me from a URE, should one occur? (ie. Can I buy URE^14 drives and sleep at night)
- Can TLER be turned off easily if I buy drives with it, and is it necessary to do so, or should I not care?
- What drives do people run successfully with RAIDZ?

(Not looking for backup advice.)
Thanks for any thoughts.
 
Hitachi Deskstars or WD Red just because of the cheaper price. I wouldn't be concerned about the failures of RED because of a little bit of stress testing. Don't buy them in one lot ;)
 
My experience has been best with hitachi desktop drives. Never given me any trouble in RAID-applications.

Why would you want to not have TLER?
 
TLER is a good thing, if you don't want your server to hang.

If you don't run a redundant setup (mirrors or raidz*), then TLER drives would be a bad thing, cause the disk will give up, assuming it was in a raid, and could be recovered using a different disk. So if your running a single disk zpool, then you wouldn't want tler, as you would want it to attempt for all it's life, to recover the data.

But the point of tler, is to give up quickly, cause you can recover the same data using another drive.
 
raidz1 will protect you from URE's just fine. I wouldn't even worry about a 4tb pool and URE's on raidz1.

I would never actually do that, cause it won't protect you from a disk failure + a URE. This is what raidz2 will protect you from.

raidz2 gets iffy on large disks, cause if you loose a disk, and during the rebuild loose another disk, then a URE would affect you. Why it's being recommended to use raidz3 with 3tb and larger disks these days.
 
patrickdk said:
TLER is a good thing, if you don't want your server to hang.

If you don't run a redundant setup (mirrors or raidz*), then TLER drives would be a bad thing, cause the disk will give up, assuming it was in a raid, and could be recovered using a different disk. So if your running a single disk zpool, then you wouldn't want tler, as you would want it to attempt for all it's life, to recover the data.

But the point of tler, is to give up quickly, cause you can recover the same data using another drive.
Click to expand...

The TLER is what, 5-6 seconds of trying to recover? Would be interesting to see how many errors are not recoverable after 5 seconds, but are recoverable after over 5 seconds.
 
ATWindsor said:
The TLER is what, 5-6 seconds of trying to recover? Would be interesting to see how many errors are not recoverable after 5 seconds, but are recoverable after over 5 seconds.
Click to expand...

I think it's 7 seconds. I'm using Reds in non raid and not too concerned about TLER, because if a drive has to "try" for 7 seconds than that's an eternity already and there might be bigger problems than just a bad block. Of course this is nothing more than my opinion, but so far so good :)

Edit - I probably should have clarified that I'm using them in an NAS. Also, there are quite a few people over at SPCR using them as non raid desktop drives and they reached the conclusion that there's really little to be worried about. WD has also commented saying there's no reason not to use them in a non raid environment and that there will be no performance decrease when used this way. Again, I think the whole 7 seconds vs 30 seconds is something that one doesn't need to be concerned about for a variety of reasons.
 
Last edited:
This guy;
http://hardforum.com/showthread.php?t=1500505
advises against TLER with ZFS. Does the concensus here disagree with that?

And I've seen *plenty* of warnings against 10^14 URE drives, saying to go for 10^15. Edit/ is the concensus, again, to not worry about it when you have RAIDZ2?

According to the current lineup of hard drives, these recommendations clash; there are no 10^15 drives without TLER, or at least not many.

I'll be doing two builds; one with 6 disks, one with 8, RAIDZ2 on both. Disks will likely be 3TB.
 
the_b_man said:
This guy;
http://hardforum.com/showthread.php?t=1500505
advises against TLER with ZFS. Does the concensus here disagree with that?

And I've seen *plenty* of warnings against 10^14 URE drives, saying to go for 10^15. Edit/ is the concensus, again, to not worry about it when you have RAIDZ2?

According to the current lineup of hard drives, these recommendations clash; there are no 10^15 drives without TLER, or at least not many.

I'll be doing two builds; one with 6 disks, one with 8, RAIDZ2 on both. Disks will likely be 3TB.
Click to expand...

My thoughts:

I din't disagree with what he is saying, an I don't think anyone else here does either. If you can set TLER off, by all means, do that, I just don't think that is very important, I think the gain is minimal, I would put it far down on the list of priorities, with the amount of good HDDs being pretty low. 10^15 would be more important IMHO, on the other hand when backblaze tested, enterprise drives had failiure rates at a similar level of desktop drives. I would get high quality desktop drives (Hitachi), or eventually high quality enterprise-drives, and be less concerned with these stats.
 
That url/guy says the same thing I did. If you care about your server being frozen, you want TLER, if you don't care, don't use TLER.

So if this is for a home build, sure, disable TLER.

If this is for a vmware datastore? defently get TLER.

Personally I perfer TLER on all disks, as nothing is that important for me to wait 60seconds for.
 
I disagree with him.

You're running ZFS. In RAID. It has another copy of the data. Why would you ever want to let the drive crunch away trying for more than 7-8 seconds to recover a block of data (which after all that time it may very well still fail to manage to read) when you've got another copy or are a quick parity calculation away from recovering it yourself and writing it to a new spot on the disk?

It is true to say that ZFS is going to better survive non-TLER drives than most other traditional RAID cards and setups, but that doesn't mean TLER isn't still both helpful and preferred if you can get it without too much extra cost.
 
I used to run WD red drives, running Seagate NAS 5900 rpm drives now. They are a quicker than the Red drives.
 
Thanks everyone for the thoughts. To conclude:
- TLER is desirable, not necessary, but neither is it a hindrance, on a RAIDZ.
- 10^15 is preferable, but I'm ok to use 10^14 disks since RAIDZ2 will insulate me from day-to-day mis-reads

Further thing NAS drives advertise is ability to tolerate vibration and heat. The guy I'm building one of the arrays for wants to use Seagate drives pulled from these http://www.seagate.com/external-har...-drives/expansion-hard-drive/?sku=STBV4000100

Locally sourced, those units are the cheapest per-TB disks available, cheaper than bare drives. The 4 TB is the same price as the 3TB WD Reds. It's a non-NAS desktop drive, 10^14, no TLER, and no claims to better handle vibration, etc. (And Seagate scored poorer on Backblaze's reliability thing recently). But, another TB per disk at the same price. Good idea?
 
It just comes down to, how much maintance and pain you want to have, when dealing with it down the road.

Oviously, more expensive, better disks, are nicer. Less downtime, less messing with it, less repairing. But doesn't mean lower quality disks don't get the job done, assuming it's ok to have download and repairs.
 
picker said:
I would consider 512n sectors desirable for any FS.
WD RED and Seagate NAS are 512e.
http://wiki.illumos.org/display/illumos/ZFS+and+Advanced+Format+disks
Click to expand...

I'm not sure you understand the article you linked to. 512e is fine when the HDD reports to the device inquiries that logical sector size is 512 and physical sector size is 4K. Are you saying that the WD Red and Seagate NAS drives currently being sold are reporting physical sector size as 512 to device inquiries? I'm not talking about advertising; I'm talking about when the HDD reports to the file system what it is.
 
We replaced all of our ZFS pools containing WD Greens and Reds with Se drives (better URE, vibration compensation mechanisms etc.) and have been quite happy with them for the past few months. They run hotter than either of the former but performance is good and the extra warranty is a consideration (5 years on the Se vs. 3/2 on the Red/Green iirc).
 
I am getting extreemly good performance out of my 512e reds.

I also just finished switching a 512 native (re4 disks) to 4k also. So future upgrades won't be an issue. Blacks have gone 4k now, so the only thing left is enterprise disks, and some of them have started going 4k.

So to continue using 512 at this time, will be really hard when you have a disk failure. Expecially if this is for a home system.
 
SE and Red's have the same URE

vi1on7.jpg


The MTBF has been upped to 1M and 1.2M if used in NAS by WD recently in their pdf doc for the SE.
 
Must be why the RE is staying 512 native instead of being AF. The last thing WD wants is their top of the line enterprise [excluding XE] is to have any compatibility issues.

So RE might be the best choice you have for now.
 
Low areal density, and Advanced Format (4K) sectors....

Hitachi Deskstar 5K4000 Specs (HDS5C4040ALE630)

Interface - SATA 6Gb/s
Max. areal density (Gbits/sq. in) - 443
Sector Size - 512e
Data buffer - 32MB
Media transfer rate - 162 MB/s
Error rate (non-recoverable, bits read) - 1 in 1014
Load/unload cycles - 600,000
Startup (A, max.) 1.2 (+5V) & 1.5 (+12V)
Idle Avg - 4.9w
Dimensions - 101.6mm x 147mm x 26.1mm
Weight (typical) - 690g
Environmental: Operating
Shock (half-sine wave (G 2ms)) 70
Vibration, random (G RMS 5 to 500 Hz) 0.67 (XYZ)
Relative humidity (non-condensing) 8% to 90%
Ambient temperature 5 to 60C
Idle (Bels, typical) 2.5
Three year warranty
 
the_b_man said:
Thanks everyone for the thoughts. To conclude:
- TLER is desirable, not necessary, but neither is it a hindrance, on a RAIDZ.
- 10^15 is preferable, but I'm ok to use 10^14 disks since RAIDZ2 will insulate me from day-to-day mis-reads

Further thing NAS drives advertise is ability to tolerate vibration and heat. The guy I'm building one of the arrays for wants to use Seagate drives pulled from these http://www.seagate.com/external-har...-drives/expansion-hard-drive/?sku=STBV4000100

Locally sourced, those units are the cheapest per-TB disks available, cheaper than bare drives. The 4 TB is the same price as the 3TB WD Reds. It's a non-NAS desktop drive, 10^14, no TLER, and no claims to better handle vibration, etc. (And Seagate scored poorer on Backblaze's reliability thing recently). But, another TB per disk at the same price. Good idea?
Click to expand...

The drives are fine but won't you lose the warranty ?
 
You must log in or register to reply here.
Back
Top