software vs hardware raid

[AMD_Gamer]

im still a bit confused here about the difference between software and hardware raid and wich is the better choice. Now from my understanding hardware raid is anything done with a raid controller card or the onboard raid that motherboards have and software raid is when you just connect the drives and use software inside of windows or linux to create the array like the disk management tools in windows server 2003. thats correct right?

now wich one is better ? i always thougt the hardware raid was better because information about the raid array was kept seperate from the hardrive that has the OS installed on it and you can take the raid card and drives and simply move them to another computer? but with software raid if you had to reformat the drive the whole raid array is lost?

 

[Sinclair]

Software is generally any device or software that does RAID and offloads its processing to a software component (driver, etc) and in turn the CPU. Most onboard RAID is of the software variety. Hardware is exactly the opposite, it does all calculations in hardware right on the card itself. For example, you have a RAID5 array, with software it would offload the XOR calculations to the CPU. With hardware, it would do those calculations right on the card using a dedicated processor. Thats the basic basics.

These days, software is generally as fast if not faster then alot of hardware RAID setups (depending on how the machine is used, only for file serving, gaming box + storage, etc), and sometimes not as fast. It also has the benefit of being able to move the entire array between machines without having to have the same hardware RAID card on each (Linux is great with this aspect).

I'm sure someone will chime in with more details, but thats the general gist of it.

 

[dirtydr]

Hardware raid is better but it comes at a cost as I'm sure you have seen if you've checked out controllers. Much better on cpu usage, you can add (larger) cache to the controllers to mitigate some of the write penalties with parity arrays, as you mentioned transplanting arrays is as easy as moving the card, etc. Many reasons to choose hardware over software.

The whole picture changes however if you are going to be accessing a machine via a 10/100 network card. If a machine is bottlenecked by a nic why not let an otherwise idle cpu stretch its legs doing parity calculations. Also, a 'classic' series processor (P2 or P3 era) will have plenty of power to get the job done.

 

[hokatichenci]

Theres really 3 types of RAID - operating system, bios, and hardware.

Operating system RAID means that the operating system takes complete control over the RAID operations, it uses no acceleration unless otherwise programmed to and is generally speaking slow. The benefit of running the RAID from the O/S is that you don't have to worry about the controller really, so long as the drives are available (ie, your onboard controller dies, you buy a pci card and plug the drives into that) it'll work. You can't boot off these arrays easily, examples are Windows Dynamic Disks and Linux md.

BIOS raid is what you usually see built into most desktop motherboards and cheaper plug in cards. These can be called "fakeraid" since they have a BIOS/cli/whatever tool that'll act like its real raid, but there is no xor offloading. They are completely controller dependent, you can't take these drives and throw them on another controller and expect them to work. Everything is proprietary. Windows can boot of these a bit easier, Linux ignores all fakeraid implementations. Sometimes these implementations will be partially accelerated.

Hardware raid is fully accelerated, offloading everything to a processor on the card. They should always show up as one single disk to the operating system. As with BIOS raid, all the implementations are completely proprietary and you usually can at best expect an array from one manuf. to maybe work on another card by the same company. More expensive hardware controllers will actually allow for multipathing, making it so that two controllers can actually work on the same RAID array - meaning that one controller can die and the other takes over. This is the costliest solution, and has the issue of array migration, but is the highest performer of all three.


With gigabit networking becoming really cheap, you'll probably still want to build a decently beefy machine (a64/p4 minimum probably) if you are planning on making a non-hardware raid server. If you are doing basic raid0/raid1/raid10, no parity calculations are involved and therefore software raid will perform a lot closer to hardware raid. P2/P3's may be fine for say 10/100 writing/reading, but if you have a large array a rebuild might take days, putting all of your data at further risk. My 8 disk 2.1tb raid5 took something like 8 or 9 hours to rebuild on a A64 3200+ using 100% cpu the entire time. I would not want to put a P2/P3 through that. Write speeds straight to the disk also cap out the CPU on that system. I get something like 120mb/s write which can fully saturate a gigabit network assuming I can push all of that throughput out to the network (NFS gets about 40mb/s). My areca raid6 array absolutely dominates my software raid in every way, shape, and form btw, aside from the controller dependence.

 

[mps]

My experiences...

Software RAID is (generally) slower and less reliable than a hardware solution.

Hot-swap support is spotty -- not all software RAID implementations support hot swapping.

Software RAID can have trouble with dissimilar drives.

The RAID implementation in WinXP (in my opinion) is flawed to the point of making it unusable. The OS must be installed before a RAID array can be created. Because of this, you can forget about any redundancy for your OS.

 

[Sinclair]

With gigabit networking becoming really cheap, you'll probably still want to build a decently beefy machine (a64/p4 minimum probably) if you are planning on making a non-hardware raid server. If you are doing basic raid0/raid1/raid10, no parity calculations are involved and therefore software raid will perform a lot closer to hardware raid. P2/P3's may be fine for say 10/100 writing/reading, but if you have a large array a rebuild might take days, putting all of your data at further risk. My 8 disk 2.1tb raid5 took something like 8 or 9 hours to rebuild on a A64 3200+ using 100% cpu the entire time. I would not want to put a P2/P3 through that. Write speeds straight to the disk also cap out the CPU on that system. I get something like 120mb/s write which can fully saturate a gigabit network assuming I can push all of that throughput out to the network (NFS gets about 40mb/s). My areca raid6 array absolutely dominates my software raid in every way, shape, and form btw, aside from the controller dependence.

There's something to be said for the controller interface being a big factor in the performance of any software RAID. We have very similar setups (8x320, 3200+) however instead of using the onboard (A8N-E, so only 4, unlike your 8) I used a 8 port PCIe 4x controller. A full 2.5TB (unformatted) RAID6 (software) array only took me about 3 hours to fully sync upon build, at about 25% CPU usage. Takes about an hour to sync a failed drive. Anything that uses the shared PCI bus (addon, integrated, whatever) with a large amount of drives is going to suffer greatly in those areas. I had tested this setup (8x320 in a RAID6) on my older P2 400 with two Promise TX4 controllers on the PCI bus prior to upgrading to the A64. It took over 24 hours to sync the array and averaged out to about 60% CPU usage. The PCI bus was just killing it. This is where some onboards cause a problem since they're bound to the PCI bus in certain situations.

 

[AMD_Gamer]

wow thanks for the help guys now im wondering about software raid and how reliable it is compared to hardware raid

with hardware raid solution all you would ever have to worry about is the raid card breaking, like stop working then you could just get a new one and be one your way? you can reformat and reinstall different OS's then just setup the raid card and you have the raid array and all your data right back?

now software raid you can have a seperate drive with the os, linux, windows server 2003 etc and you setup all the info about the raid array in the os on that drive? now if the drive fails, gets corrupted , gets a virus you lost everything on your array?

 

[unhappy_mage]

With Linux software raid, I've had good luck with regard to reliability. I've had raid arrays of most levels at some point (0145 but no 6) and have never lost an array due to problems with the raid layer. Once I lost an array because of bad sata drivers, but choose your card to be well-supported and you'll be fine.

With raid of any kind, a seperate boot disk is usually for the best.

Software raid arrays from Linux can migrate from one machine to another, or a different controller, or a different type of controller, with minimal hassle. Disk order is kept on-disk, as well as information about the raid configuration, so one can plug the array into a machine that's never had a raid array before and mount the volume and it all comes up as expected. Works quite nicely.

How much experience do you have with Linux? LSR is quite reliable if one knows what one is doing, but it's rather easy to accidentally nuke a filesystem or an array, so array management isn't recommended for a first Linux project.

 

[Sinclair]

wow thanks for the help guys now im wondering about software raid and how reliable it is compared to hardware raid

with hardware raid solution all you would ever have to worry about is the raid card breaking, like stop working then you could just get a new one and be one your way? you can reformat and reinstall different OS's then just setup the raid card and you have the raid array and all your data right back?

now software raid you can have a seperate drive with the os, linux, windows server 2003 etc and you setup all the info about the raid array in the os on that drive? now if the drive fails, gets corrupted , gets a virus you lost everything on your array?

Both are the same in terms of filesystem reliability. If you delete something, a virus goes crazy or you format the system, both can be equally trashed. The RAID "info" is generally stored in a superblock (MBR, whatever the case may be) in a software raid, so if you lose your OS drive for some reason, your RAID array won't go with it. You can format your new drive, reinstall the OS, and the array drives will still be intact.

 

[mps]

The RAID "info" is generally stored in a superblock (MBR, whatever the case may be) in a software raid, so if you lose your OS drive for some reason, your RAID array won't go with it. You can format your new drive, reinstall the OS, and the array drives will still be intact.

While this is true, I would certainly never rely on a system set up like that. Why on earth would you set up a redundant system for data and not have something similar for your OS? In the event of a system failure, your data in software RAID *might* still be intact, but how long will it take you to recover & rebuild?

I stand by my earlier comment regarding redundancy on a boot drive. With the equipment I work with (servers) on a daily basis, I would NEVER have a boot drive without some form of redundancy, which eliminates software RAID as an option. In my opinion, the only way to do it that provides maximum protection against downtime is a hardware based solution with a mirrored boot drive and RAID 5 for the data.

AMD Gamer, what is your eventual goal here? Are you looking for a cheap way to stripe data across multiple drives? Maximum uptime/minimal recovery time? There has been a lot of good facts in this thread, but I feel we are all offering advice based on our environments and usage, and that may not match up to what you need. I am using RAID at my clients for high availability and minimal downtime in the event of a failure. I need my RAID systems to support hot-swapping of failed hard drives, and on-the-fly rebuilding of data without shutting the server down. I am dealing with bleeding edge servers with redundant storage, redundant power supplies, etc. Software RAID will never work for my needs, and I will never recommend it. It has its place, but not in my world.

Just remember that whatever direction you go, you still need a good backup strategy no matter what solution (hardware RAID or software RAID) you choose.

To answer your question about replacing a failed card, some (but not all) RAID adapters support dumping the striping config to a floppy. If the adapter fails, you replace it with an identical adapter, load the config from the floppy, and you are back in business relatively quickly. If you have an adapter that supports this process, and you do not have the backup of the striping config, you are screwed. You really need to dump the config as soon as you build the array for the first time. Do it before installing the OS and store away several backups of that config. Since the new adapter does not contain the striping config in the on-board Flash memory, it has no way to properly access the array.

 

[hokatichenci]

I stand by my earlier comment regarding redundancy on a boot drive. With the equipment I work with (servers) on a daily basis, I would NEVER have a boot drive without some form of redundancy, which eliminates software RAID as an option. In my opinion, the only way to do it that provides maximum protection against downtime is a hardware based solution with a mirrored boot drive and RAID 5 for the data.

In Linux you can setup a raid1 O/S and raid5 data just fine using md. I think you can even swing some crazier setups using lvm2/evms and initrd. The best way to protection against downtime is having everything in some form of raid with hot swapping, hardware raid isn't really a prerequisite. It sure does make things easier though.

 

[uOpt]

with hardware raid solution all you would ever have to worry about is the raid card breaking,

Yes, this is a big problem. In my opinion, building a RAID for reliability and then only buying one controller is a joke. You need a second controller on the shelf.

No such problem with software RAID.

like stop working then you could just get a new one and be one your way? you can reformat and reinstall different OS's then just setup the raid card and you have the raid array and all your data right back?

If you get the same kind of controller you can just move the drives. Changes vendors or sometimes models and you need to reformat.

No such problem with software RAID. You can run software RAID over any drive connected anywhere, in all mixes (as long as it is reliable enough).

now software raid you can have a seperate drive with the os, linux, windows server 2003 etc and you setup all the info about the raid array in the os on that drive? now if the drive fails, gets corrupted , gets a virus you lost everything on your array?

The approach to boootstrap software RAID varies.

Of course you don't want to have your whole OS on a single-disk config.

In Linux the most common approach is to have a readonly /boot partition that you have a copy of on the second/third drive.

For FreeBSD I usually use a readonly / partition anyway, and again that is mirrored.