multiple small raid 10's or few big raid 10's, what's better performance wise?

Red Squirrel

[H]F Junkie
Joined
Nov 29, 2009
Messages
9,211
My file server / NAS which everything rides on from raw file access to VMs uses md raid, I currently have a 4-disk raid 10 with 3TB drives (Toshiba/Hitachi), and a 8-disk raid 10 (WD reds) and there's a 10-disk raid 5 but it's only for backups as I moved stuff off it.

I find I often get lag/performance issues where the load average goes as high as 10 or even higher at times. During these times the system is practically unresponsive and then everything on my network takes a hit. This is especially true during backup jobs or other operations.

I'm looking at increasing performance, just wondering, would I be better off just making a larger raid 10? Or am I actually better off splitting stuff up in smaller luns? I am thinking of retiring the raid 5 array and that will give me about 9 extra bays that I can put bigger drives in to do one or more raid 10's or expand existing raid 10's.

What's the better route to go for better performance?

I'm also considering ZFS on Linux, would I get better performance than md raid with that?
 
I
- use SSD only pools where performance is needed
- add a dedicated ZIL device when secure write behaviour is needed like a ZeusRAM or Intel S3700
- prefer double parity raid Z2 or triple parity Z3 for the rest

ZFS as a filesystem is build for data security not performance in the first place.
Due the extra checksums it cannot be as fast as a checksum less and simpler filesystem like ntfs or ext4.

But ZFS includes a volume management, raid management, pool management and share management paired with copy on write (always consistent filesystem with snaps). These features alone would be worth a performance degration.

But if you have enough RAM an additional ZFS feature can give you a superiour performance with ZFS. This is the ARC cache paired with an optional L2ARC Cache on SSDs. Free RAM is used as intelligent readcache. ZFS ARC is one of the best available caching mechanism. With ARC it is quite usual that 80-90% of all reads are deliverd from RAM after some time.
 
I second what _Gea says. Just be careful with L2ARC if performance is a concern, L2ARC will use some of your RAM to operate, which means a smaller ARC. ARC will always be faster than L2ARC.

If you consider spending money on an SSD for L2ARC, buy more RAM instead. Only when all RAM slots are filled with the biggest sticks money can buy should you consider L2ARC.
 
Ok but if I stick with md raid, what's the best performance wise? lot of small raid 10's or try to have a single or few big raid 10's? (more spindles).

I have lot of issues where everything grinds to a halt, especially when there are backup jobs running.

What about raid 6, am I better off having a giant raid 6 instead of raid 10's?

This is mostly for VMs and general file access.

I will be retiring one of the older raid arrays which is raid 5, that will give me 9 extra slots, so I can either make more raid 10's or expand the existing raid 10's.
 
I recently got a RAID10 using 20 1TB disks. From my understanding, the name of the game when it comes to performance is either more spindles or faster SSDs.
 
Oh and I went with RAID 10 because I wanted a good blend of speed, redundancy and fast rebuild times in case of disks going bad.
 
Back
Top