Possibly building RAID 5 - Good company?

[shadowwyvern]

I *may* finally be building an 900GB (4x300GB) raid 5 array, and have a few questions. Currently, I am looking at these items:
(4) Seagate 7200.9 300GB drives (here)
(1) Istar BPU-340SATA backplane (here)
(1) SiliconImage 3114 SATA controller (here)

(I will be buying that controller from the US from the same buyer, who seesm to have 100s of them, but currently that auction has the longest remaining time)

1.) What really is a good company to buy drives from? On Newegg, practically every single drive I have looked at has reviews for of horror stories about DOAs, 3 month drive lifespans, etc? My personal conclusion is that at elast 70% is personal opinion/luck. I do not want to put 900GB of data at any risk whatsoever (hence the raid 5)

2.) Anythign wrong with my controller? I can't afford anythign much more expensive, and do not need a huge array of features.

 

[venm11]

what are your requirements/objectives for this raid array? What's the system for, and what other hardware is going into it?

I did quite alot of research before building mine and ultimately decided against raid 5 because the total system cost to get the desired performance went way too high to get serious benefits. For example, using a raid-5 card on a 32-bit/33mhz PCI bus with software XORing is not going to be stellar. Also, I noticed that raid 5 is statistically more risky for data than using 1 drive; I guess the parity just mitigates *some* of the risk that's increased when using multiple drives. And that's just to make high availability possible; you still need a backup plan.

Sorry if i'm sounding preachy... I had quite a few frustrations from not having the right setup. I also listen to others' headaches at work when drives fail or raid controllers misbehave... or whatever.

 

[feigned]

(1) SiliconImage 3114 SATA controller (here)

2.) Anythign wrong with my controller? I can't afford anythign much more expensive, and do not need a huge array of features.

http://www.newegg.com/Product/Product.asp?Item=N82E16815124020

Look familiar? Same card, just flash it with the BIOS from the Silicon Image site and you'll have a RAID 5 capable card on the cheap.

Honestly though, if you're going to do it you might as well dish out the proper bucks for a decent controller.

 

[hardwarephreak]

I can understand your reluctance about drives dying prematurely...but two things to remember...

1. Everyone has different experiences...some people swear by a particular brand, some just swear at them.

2. You know, apart from the inconvenience, you ARE building a redundant array, if a drive fails you should be covered.

 

[WoodiE]

1.) What really is a good company to buy drives from? On Newegg, practically every single drive I have looked at has reviews for of horror stories about DOAs, 3 month drive lifespans, etc? My personal conclusion is that at elast 70% is personal opinion/luck. I do not want to put 900GB of data at any risk whatsoever (hence the raid 5)

I've heard the same thing about Newegg, in fact there was a thread not to long ago on here about their poor shipping. Someone from newegg came here and said something along the lines that they would look into improving their shipping methods but that's the last I've ever heard about it.

Personally I've ordered a few drives from newegg and only one had to be RMA'd after a few months.


-Michael

 

[unhappy_mage]

ZipZoomFly (and HyperMicro, but their selection stinks) are currently the champs of hard drive packing. They use foam instead of bubble wrap, Here's the drives you linked.

As for a controller, the one someone linked above will be good in Linux, but I wouldn't trust it in Windows. Linux software raid is device-independent, so you could even use onboard sata ports with it. If using Linux as your only OS doesn't strike your fancy, you could set up another machine and use it as a file server. It'd be a good deal more expensive - it's another entire machine, after all! - but it's a lot more flexible, too. Something to consider.

 

[shadowwyvern]

This machine is currently my primary Windows desktop, but will be converted to a linux server in about 6 months. From what I am reading though, is the card I have chosen a true hardware raid device? If not then I guess I should shell it out for a true hardware one, otherwise it is simply not worth it. Any good sugestions for raid 5 contriollers that are decent that use the 32 bit PCI bus, or PCIe 2x or 16x (desktop mobo, am I correct in thinking that the 16x slot will accept card of any size up to 16?). My budget for a card could run as high as $150, but much more than that is really pushing it. Although I can use my 16x PCIe, I would rather not as it means I qould be unable to get the setup working anytime soon, as my graphics card currently has sole reign over that slot. It only needs to be a 4 port, I won't have any room left in my case after the 4 drives anyway.

I don't really care all that much about performance, my curent large data drive is a USB one, and the machine in it's ultimate role will be a network file server. Reliability takes precedence over speed here (but I do not want raid 1)

 

[unhappy_mage]

That's not true hardware raid, and you're not going to touch hardware raid for any number of ports for under $300 or so, and I would be hard pressed to recommend that card for anything. Since you're not concerned about performance, though, stick with this card. It's well-supported under Windows and Linux, can't beat the sil3114 for genericity.

 

[venm11]

This machine is currently my primary Windows desktop, but will be converted to a linux server in about 6 months. From what I am reading though, is the card I have chosen a true hardware raid device? If not then I guess I should shell it out for a true hardware one, otherwise it is simply not worth it. Any good sugestions for raid 5 contriollers that are decent that use the 32 bit PCI bus, or PCIe 2x or 16x (desktop mobo, am I correct in thinking that the 16x slot will accept card of any size up to 16?). My budget for a card could run as high as $150, but much more than that is really pushing it. Although I can use my 16x PCIe, I would rather not as it means I qould be unable to get the setup working anytime soon, as my graphics card currently has sole reign over that slot. It only needs to be a 4 port, I won't have any room left in my case after the 4 drives anyway.

I don't really care all that much about performance, my curent large data drive is a USB one, and the machine in it's ultimate role will be a network file server. Reliability takes precedence over speed here (but I do not want raid 1)

True raid: you can get 3ware cards on ebay for $100-200 or so, Raidcore (not raid 5 exactly, but good enough) for about $150. For pci-express, the Areca cards are supposed to be pretty good, but they're all 8x, but some SLI boards may have trouble running them in one of the 16x slots. 3ware's pci-e cards are on the verge of coming out.

You could run a pci-x/64-bit card in a 32-bit slot, but don't run any other high-bandwidth pci devices like gigabit, etc.

 

[shadowwyvern]

How compatible are PCI-X cards with normal PCI slots? And I beleive my gigabit ethernet runs on the PCI bus as well, but they seem to have come out with PCIe ethernet cards that are supported in linux.

What is the real disadvantage of non-true hardware raid? How does it differ from strictly software raid, and is it dependent upon system stability (I recal a friend who did not want soemone to attack his linux box in a hackign war due to his concern over his software raid). If it is just processor usage to do the XORing, then I am not really concerned, as this machien will have power to spare as a server, but if the array could be put in any extra jeopordy by system crashes than a single drive, forget it.

My board is not an SLI one, it has only a single PCIe x16 slot, which has ntohign special about it as far as I know.

I am very interested in raid 6 (not that I am quite that paranoid) though

 

[venm11]

How compatible are PCI-X cards with normal PCI slots? And I beleive my gigabit ethernet runs on the PCI bus as well, but they seem to have come out with PCIe ethernet cards that are supported in linux.

generally pci-x cards are completely compatible with 32-bit slots... I haven't had any that didn't work (that i've tried). Of course... obviously they were designed to use 2-6 times the bandwidth. Looking at a block diagram for the chipset/mobo can say alot about how best to avoid bottlenecks. Probably, if you have have pci-e, that should be fine.

What is the real disadvantage of non-true hardware raid? How does it differ from strictly software raid, and is it dependent upon system stability (I recal a friend who did not want soemone to attack his linux box in a hackign war due to his concern over his software raid). If it is just processor usage to do the XORing, then I am not really concerned, as this machien will have power to spare as a server, but if the array could be put in any extra jeopordy by system crashes than a single drive, forget it.

there's a couple of points. first, with a hardware raid card, the array is maintained by the controller. So, if you were to plug that card (and drives) into another machine, or change the OS, the array should survive; on a software solution, it's going to be alot more difficult because the array configuration needs to be moved to the new OS (assuming it's the *same* OS, same libs, etc).

Second point is performance... and this is where I get fuzzy. AFAIK, a hardware raid card can read stripes in parallel across the drives, XOR them (which cuts down data by ~25%) on the fly, and feed them across the pci bus as one "chunk". On a software raid setup, they are read in series across the PCI bus and then combined and XORed in ram, so there's latency hit with each operation. What does all this mean in real terms? I have no idea.

I am very interested in raid 6 (not that I am quite that paranoid) though

Raid 6? Don't know any ata/sata controllers that support that. It's not clear why you need the extra redundancy. I can't emphasize enough the value of having a good backup solution, though. RAID only does so much.

 

[unhappy_mage]

there's a couple of points. first, with a hardware raid card, the array is maintained by the controller. So, if you were to plug that card (and drives) into another machine, or change the OS, the array should survive; on a software solution, it's going to be alot more difficult because the array configuration needs to be moved to the new OS (assuming it's the *same* OS, same libs, etc).

Linux software raid is designed to be controller independent. In fact, it's pretty much anything-independent - you can use the same drives with pata->sata controllers and an sata controller, or a scsi->sata enclosure, or whatever, as long as it's a block device and the bytes are there. Anything else... you're on your own. If it's a pci card, move the whole thing (make sure to label cables!) and it should work. LSR, you don't even need to label anything. Plug in enough disks, regardless of order, and you get an array.

Second point is performance... and this is where I get fuzzy. AFAIK, a hardware raid card can read stripes in parallel across the drives, XOR them (which cuts down data by ~25%) on the fly, and feed them across the pci bus as one "chunk". On a software raid setup, they are read in series across the PCI bus and then combined and XORed in ram, so there's latency hit with each operation. What does all this mean in real terms? I have no idea.

Well, latency across the PCI bus will be the big factor. Suppose you've got an array with 4 drives, 64k stripe size. You've been doing a full-stripe, sequential write, so the disks are already aligned (more or less). With a hardware card, the machine will: receive a write request of 3N size from the system, send it over PCI bus (done talking to hardware at this point), hardware card does XORing and issues 4N worth of writes in parallel. With a software array, the machine will: receive a write request from the OS, do the XORing in software, write 4N over pci bus (done talking to hardware at this point), and the firmware of the card will take care of sending out the write requests to the disk. So the difference on a full-stripe write is only (N+1)/N writes over PCI bus, and XORing in software. Vanilla PCI is 32 bits at 33 mHz, for ~1gbit/sec. So writing the extra 1 64K chunk takes about (64*8*1024) / 1e9 ~= .0005 seconds. Compared to the XORing of 64k worth of data (64k bytes / 4 bytes/word = 16k ops, worst case - MMX or SSE will probably optimize this) at say 2gHz = 16k / 2e9 = 0.000 008 seconds, pci is definitely the slowdown here. It takes about 64 times as long to transfer the data over PCI as it does to figure out what to transfer. Granted, OS scheduling may interfere with this, but the XOR engine will be at a very high priority in any logical system; you can dither for a long time before you take enough CPU that the PCI bus isn't the primary slowdown.

However, this is a best case. Full stripe writes don't happen all that often. For non-full-stripe writes, it's significantly worse for software raid. I'll let you work that one out, though

Raid 6? Don't know any ata/sata controllers that support that. It's not clear why you need the extra redundancy. I can't emphasize enough the value of having a good backup solution, though. RAID only does so much.

Areca controllers support this.

OP, WRT stability of software raid, I'd imagine your friend was concerned more about the data being erased than a stability issue. I've run LSR of some form for ~3 years without any problems.

 

[venm11]

unhappy_mage- interesting post. Personally, I had enough headaches with simple LVM to not even attempt linux software raid. Can you recommend a good LVM and/or soft raid graphical configuration tool? I've been doing everything manually.

Having a hard time translating the theoretical figures into real-world performance. My file server is actually a (*drum roll*) 2x733mhz p3, although it does have pci-x (1 3ware card + 1 intel 2xGbe x 1 scsi). What sort of cpu utilization hit do you think i'd have with software XOR? (I'm thinking of adding a software array of scsi drives). How much for a top-end system?

OP, WRT stability of software raid, I'd imagine your friend was concerned more about the data being erased than a stability issue. I've run LSR of some form for ~3 years without any problems.

We might be on the same page about that. I was trying to make the point that raid parity was more about high availability (drive failure) than protecting data, and that raid doesn't replace the benefits of backups.

 

[unhappy_mage]

evms is my hero My file server's only a pair of 866 p3s, so you're not much behind. I only get 5% or so cpu hit at worst, and it fluctuates a good bit - writing at constant speed as shown by "vmstat 1" goes from half a percent cpu to five. Your guess is as good as mine as to why.

I want one of those Intel nics. And maybe another pci-x slot so I'll have somewhere to put it

Note that if you've got raid 5 built into your kernel you can see what kind of speed you can get through your processors (if it's built as a module, try "insmod raid5" and it'll do the same thing) - try "dmesg | grep -i raid" and you should get back some stuff about some large number of megabytes per second. On my system it's ~1800; I'd post the exact number but, since i've had my system up for 6 weeks, shorewall has long since filled my log with stuff about packets it's blocked. I should really turn off logging.

 

[shadowwyvern]

I have decided to bite the buttlet and ebay a 4port SATA 3ware/areca controller. I will not be building it in the near future though the addition of a decent controller has put it over the cost threshold, for now.

As for backup, I will buy soem tapes and beg/borrow a tape drive off someone to do backups. Even with the small amount of data I have now, DVD backups are too labor intensive.

 

[paradoxblue]

seriously. tape back up. how much data can a tape hold :| i havnt ever seent hem being larger then dvd-r "S or maybe thats just me.

edit===
i stand corrected. guess they got 100-200gb tapes. the tapes are cheap but the drives seem to cost almsot as much to jsut buy a second set of hard drives now lol. is there a feesible methode for backing up 250+ gigs yet?