Non-enterprise drives in software raid..

[tweak2]

Well, here's the deal: 34x 2Tb sata drives on 2 controllers (set up as non-raid) and put into a soft raid 5 with 1 hotspare array in linux (likely centos).

Firstly, should I be concerned about using non-enterprise class (ie: no TLER) if it's not on hardware raid? There will be a good amount of i/o on these drives, but you save about 34*$120 by not using enterprise drives in this environment...

Secondly, would the 64Tb array size be a huge problem? Any kernel/fs tweaks required to get it there?

Any thoughts or experience with this?

 

[Red Falcon]

You don't need TLER in software RAID.

However, you should at least do RAID 6 as it will allow up to two drives to fail.
Absolutely do not do RAID 5 unless you want to see your array go bye-bye.

Still, with an array with that many disks, any RAID would be a bad option and ZFS should be used.
At a minimum, if your controllers can handle it, go with RAID 60, at least it will have double parity over two array containers.

I've seen people with multiple drives fail at once and their arrays are destroyed due to it, and even with RAID 6, if three drives fail (even during rebuild), your array will be lost.

You should honestly go with Solaris and ZFS (RAID+fs)

The size of the array should not be a problem.

34 drives are going to create a lot vibration, which may hamper performance considerably if desktop-class drives are used. There is a reason why nearline-class and enterprise-class drives exist, they can withstand that kind of punishment.

Another thing, make sure there is ample cooling on the drives.
Desktop-class drives cannot begin to handle that kind of heat if they are in close-quarters with one another and you will have disk failures. The best thing to do is have lots of cooling, fans, containers, AC in a basement-level room, etc.

Software RAID is very good and flexible, but with that amount of disks and the huge amount of parity, I honestly believe ZFS would be the better choice, both for protection and performance.