• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

software raid 5 reliability (linux)

E

eth00

[H]ard|Gawd
Joined
Aug 16, 2004
Messages
1,090
I am probably going to get a killer deal on some 400gb SATA drives, around $100 no rebates :), from a friend and am looking at building up a fileserver. This is just going to be a home fileserver so I do not care about any of the transfer stats since I am going to be doing it all over a 802.11G network and that will limit me enough. I am looking at 4 drives at least and possibly 6+ drives which which is going to drive it into a fairly expensive controller if I go the hardware route.

The thing I am most worried about is reliabtiliy. I have heard horror stories about windows arrays dumping and loosing everything. While I know it is just as possible with linux has anybody done a software RAID 5 array and had any serious problems with it that make it completely not worth it ?
 
I'm currently using Linux software RAID 5, with a total of 3 Maxtor 120GB PATA drives. The only problem I've had to date has been cable related. The rounded cables I was using to connect the drives to the controllers were causing CRC errors, and the kernel kicked one of the disks out of the array because of it. The disk itself was fine. To keep from having to deal with the nightmare of ribbon cables, I bought some SATA converters and a 4 port SATA controller. Once had the cable issue taken care of, I added the disk back to the array, and all has been well since. Now I have decently clean wiring, and an upgrade path :D
 
I'm using 3x300gb disks with evms in raid 5, I've successfully dropped and restored a disk. If you're willing to get some cutting-edge hardware, you can get an AOC-SAT2-MV8 8-port sata card for around $110, which is pretty good if you ask me. It's supported fine under PIO, and DMA support is on the horizon. Cheap, and slow right now, but with potential to be fast.

 
If you are willing to split the array, I have seen some very cheap SATA 4 port raid 5 cards on ebay (around $35US plus shipping).
 
One Gentoo server I have was having problems with software raid5, I ended up going with raid10 and I've successfully tested dropping drives in that mode. It was 2.6.9 that I was having problems with so maybe whatever issue it was has disappeared by now, but for some reason the drive would never become "clean", it would always show one bad drive and indicate hardware errors when there were none.
 
I am very non-believing in Linux software RAID.

After having to clean up the mess of another tech who upgraded a box from RH8 to RH9. The drivers for the Silicon Image ATA controllers were vendor-supplied when RH8 was installed but RH9's kernel included (buggy as all hell) drivers for it. Upon reboot into RH9, the new drivers resulted in seek errors on writes causing two out of four of the array members to drop offline.

It was a 4x120GB array, btw.

So I had to DD each drive across the network to my workstation to give me a margin of error. After speaking to the developers of the software RAID tools, all that had happened was the checksums had changed. Somehow. The only solution was to delete and re-create the array.

I have never lost data on a Windows soft-RAID array but Linux was a close-call.
 
So that's not a LSR problem, that's a linux driver problem. Not to mention a lack-of-backup problem. If you'd been using a well-supported chipset, there would have been no problem.

 
unhappy_mage said:
So that's not a LSR problem, that's a linux driver problem. Not to mention a lack-of-backup problem. If you'd been using a well-supported chipset, there would have been no problem.

Click to expand...

Yeah honestly, if you use crap hardware in Windows you'll get crap results there too. Linux doesn't magikly make shitty hardware into good hardware, but it's a great option if you're willing to put a little effort into buying compatible hardware.

I've been running a multiple terrabyte software RAID below LVMS for two years and haven't had a single byte of data lost. One of the Seagates did die but I was up and going within a week with my new replacement.
 
Yeah I am not so concerned about the driver part, I run a pretty sucessful linux adminstration business and I can source compile a kernel tweaked out for a fileserver along with the latest drivers if needed.

I have seen the cheap (under $50) RAID cards but my opinion of them is pretty low, I am thinking that it would be more reliable to just go with a software RAID setup vs a cheap hardware one.

One of the major "problems" with RAID 5 is that it requires a lot of cpu overhead but since there would only be 2-3 people on it max at a time and we are not requiring blazing fast speeds that is not a problem. My primary concern is that I want a large space where I can backup pictures and home movies and not hae to worry about the array crashing.

I think I will take a look at that sata controller with 8 SATA connectors if I want more then the default 2 (or maybe 4) that most boards have, that would be exactly what I need if I just end up going software raid :) I don't think at this point I want to spend $500+ on a decent controller to most likely only get better speeds.
 
eth00 said:
Yeah I am not so concerned about the driver part, I run a pretty sucessful linux adminstration business and I can source compile a kernel tweaked out for a fileserver along with the latest drivers if needed.
...
I think I will take a look at that sata controller with 8 SATA connectors if I want more then the default 2 (or maybe 4) that most boards have, that would be exactly what I need if I just end up going software raid :) I don't think at this point I want to spend $500+ on a decent controller to most likely only get better speeds.
Click to expand...
I have a working .config for 2.6.14.5; as *soon* as the DMA support gets enough testing to be out of beta, I'll have one for that.

I can attest to the quality of the card; if you want pics, specs, benchmarks, whatever, let me know and I'll see what I can do. Supermicro did a tremendous job with this card, and I'd like to see them become more popular. I got mine from wiredzone.com quickly and with no problems. The price has gone up to $132 there, but if you're willing to try another company exshopping.com has it for $125 shipped.

Happy customer unhappy_mage, reporting for duty :D

 
unhappy_mage said:
I have a working .config for 2.6.14.5; as *soon* as the DMA support gets enough testing to be out of beta, I'll have one for that.

I can attest to the quality of the card; if you want pics, specs, benchmarks, whatever, let me know and I'll see what I can do. Supermicro did a tremendous job with this card, and I'd like to see them become more popular. I got mine from wiredzone.com quickly and with no problems. The price has gone up to $132 there, but if you're willing to try another company exshopping.com has it for $125 shipped.

Happy customer unhappy_mage, reporting for duty :D

Click to expand...


Nice that makes me feel good inside :) I actually have a .config back from 2.6.9 that is stripped to the bare min to boot then I tack on the hardware as needed, so far still going strong for 2.6.14.x. Knowing you are running both seems to be a good indiciation though, ti looks like software raid for me :D The $130 for a controller is not horrible when I am looking at $500-800 for a hardware controller...now I just hope the 400gb drives for $100 comes though and I will be golden.
 
eth00 said:
I am looking at 4 drives at least and possibly 6+ drives which which is going to drive it into a fairly expensive controller if I go the hardware route.
Click to expand...
Pretty sure all ASUS NF4 boards support hardware RAID 5, so maybe just get one with 6 or 8 SATA ports and go that route? Might be cheaper than a controller card, might not.

EDIT : I was wrong. ASUS has two boards that support RAID 5, they are the A8N32-SLI Deluxe and the K8N4E-Deluxe
Neither have more than 4 ports supporting RAID 5, however, plus the K8N4E only supports RAID 5 through the SIL3114 controller, so it's a 150MB/S limit for you, no NCQ or anything like that.
On the other hand, apparently the nVidia 6100 and 6150 chipset supports RAID5 natively, and I'm pretty sure nVRaid can do arrays across multiple controllers, so you might be able to find some sort of board that has an extra controller on there so that you can have more than 4 ports. Just an idea.
 
banGerprawN said:
EDIT : I was wrong. ASUS has two boards that support RAID 5, they are the A8N32-SLI Deluxe and the K8N4E-Deluxe
Neither have more than 4 ports supporting RAID 5, however, plus the K8N4E only supports RAID 5 through the SIL3114 controller, so it's a 150MB/S limit for you, no NCQ or anything like that.
On the other hand, apparently the nVidia 6100 and 6150 chipset supports RAID5 natively, and I'm pretty sure nVRaid can do arrays across multiple controllers, so you might be able to find some sort of board that has an extra controller on there so that you can have more than 4 ports. Just an idea.
Click to expand...

I would be really weary of doing hardware RAID across mutliple and different devices, I think I will avoid that for now. Thanks for the link to the boards that support RAID 5 natively though that is a real option. IThe 150MB/sec is not a huge deal since it is only a lan. That looks like a really good way to go if I decide on "only" 4 x 400gb array.
 
You must log in or register to reply here.
Back
Top