Question about Software RAID

[yashi]

Hi

I want to protect my data from disc failures. But I'm unsure what the best solution in my case is. Either i build a dedicated fileserver, or add the storage to my main PC.

I dislike the first version, because it would need unnecessary energy, i only use the files on one PC anyways.
I dislike the second, because I'm not sure how a RAID behaves if i have to format my main drive. Can i just tell Snapraid f.e. 'Here are the drives, make it usable again!' ? Or do i have to backup special config files to be able to reuse the array?

Also how is the scalability? Can i just replace f.e. 1tb drives one by one with 2tb, and end up with double the available space? Or simply add new ones to the existing array?

Atm. im talking about 5-10TB.

 

[ashman]

What are you currently doing for backups, if anything? You could always grab some external USB drives, I just saw a 5TB model advertised the other day.
There are a few variations of linux software raid available, I am by no means an exert in them, however they do allow you do use drives of various sizes and you can replace each drive when it fails, usually you only use whatever data is on the drive that failed, in a worse case scenario and you can also expand by adding drives on the fly as well.

 

[drescherjm]

What is the purpose of this? I mean for media files that only get added and rarely if ever get updated and may get deleted snapraid will allow you to mix and match drives adding more drives any time you want keeping each drive as a separate drive formated with whatever filesystem you use on your computer.

For example for my linux based htpc I have 6 data drives and two parity drives. 5 of the 6 data drives are 2TB drives 2 of these have a zfs filesystem, 2 of them have btrfs and the last has ext4. The 6th data drive 4TB has zfs on it as well. Note that none of these disks are combined in any way. The 2 parity drives are 4TB externals. These only need to be powered on when I sync, scrub or check the snapraid. Also these disks are in no way connected when it comes to reading and writing. 1 disk is all you need to access the files on that disk. You can boot with only 1 disk and happily use the files on that 1 disk or take any of the drives to a different machine and use the disk as a single disk.

If the purpose of this extends beyond media to files that change often SnapRaid is not a good fit for the data that is rapidly changing.

 

[hrana]

Hi

I want to protect my data from disc failures. But I'm unsure what the best solution in my case is. Either i build a dedicated fileserver, or add the storage to my main PC.

The advice above is good. If you want to protect data then a good backup strategy is your only option. RAID is for data availability not backup. With that said, is there some reason you prefer snapraid over other options?

 

[yashi]

The advice above is good. If you want to protect data then a good backup strategy is your only option. RAID is for data availability not backup. With that said, is there some reason you prefer snapraid over other options?

The reason i wrote snapRAID is based on this chart: http://snapraid.sourceforge.net/compare.html
All others seem to have downsides.

>RAID is for data availability not backup.

It does protect me from drive failures, which is my main concern.

 

[mrkazador]

I use snapraid on my mediaserver and not too long ago I added a ssd for my os drive. To get snapraid running again was fairly simple. All I did is edit my snapraid.conf (settings file) to point to the drives as I had it before the format and everything was working great again. You do want to keep multiple "content" files on separate disks, this is your precious raid info.

You can replace a drive with a bigger one or add to the array with no problems. All you need to do is edit the snapraid.conf to point to the data drives and you're done.

 

[yashi]

due to all the posts on the forum i totally focused on software solutions like zfs, unraid, flexraid etc...

Now im wondering, whats so bad about the buildin RAID feature which every mainboard has nowadays? Im guessing that all mainboards with more than ~6 SATA interfaces cant create HDD configurations with more than 6 drives, because they have two sata controllers? And i wont be able to create a raid that makes use of both controllers at the same time?

 

[cashkennedy]

You cannot create hardware raid between drives on 1 controller and drives on another controller. The intel controller could support 6, 8, or 10 drives depending on the model. Not all intel motherboards support raid (this is how they differentiate the cheaper chipsets from the nicer ones).

 

[Dangman]

due to all the posts on the forum i totally focused on software solutions like zfs, unraid, flexraid etc...

Now im wondering, whats so bad about the buildin RAID feature which every mainboard has nowadays? Im guessing that all mainboards with more than ~6 SATA interfaces cant create HDD configurations with more than 6 drives, because they have two sata controllers? And i wont be able to create a raid that makes use of both controllers at the same time?

The main issue is that they're just really slow in terms of performance in comparison to the majority of software and true hardware solutions out there. In addition, they lack the various extra extremely useful/neat features you see with software solutions. Finally, there's the appearance that they're not quite as reliable for parity RAID like you see with both true hardware RAID solutions and software solutions.

AMD's FM2 mobos are some of the only consumer-grade mobos where they have 8 SATA ports controlled by the same storage controller. But alas, it is an AMD controller. At least in the past, AMD's storage controllers weren't as fast nor as reliable as Intel's storage controllers. That may not be the case today but still something to keep an eye out for. However, with mobos using two different storage controllers, no you can't make a RAID array using both storage controllers unless you're using a software solution like ZFS, Linux RAID, FlexRAID, etc.

 

[Jagger100]

I like SnapRAID and some of the other nonRAID alternatives. This is because ZFS and hardware RAID are enterprise solutions with enterprise budgets and resources.

For example, ZFS and hardware RAID protect against harddrive failure, but do they protect against Motherboard, powersupply or memory hardware failures? How easy is it to recover my data? If I have exact hardware lying around, its possible. With some of the alternatives I can just pull the drive out of the home server and put on a USB adapter or mount it to a sata port and I have my data again in a couple of minutes.

I think the catestophic Server failure where things other than the HDs are damaged or a large number of HDs get damaged without backup hardware handy for box rebuilding is much more likely at home. The cat peeing into the server box scenario.

I also can upgrade my home server without having to keep the old server alive. With ZFS, want to get a new box and move your data over, you need the new server running simultaneously with the old or a temporary backup option to hold the data.

Its true SnapRAID is somewhere between a RAID/backup system and not a full RAID but as long as I keep a copy of the data on my local PC until the next snapshots, I should be covered.

And also SnapRAID isn't as urgent to have an ECC memory & capable keyboard which is more cost friendly to home.

 

[SirMaster]

For example, ZFS and hardware RAID protect against harddrive failure, but do they protect against Motherboard, powersupply or memory hardware failures? How easy is it to recover my data? If I have exact hardware lying around, its possible.

I also can upgrade my home server without having to keep the old server alive. With ZFS, want to get a new box and move your data over, you need the new server running simultaneously with the old or a temporary backup option to hold the data.

With ZFS you can export the zpool and import it to another computer. The hardware does not matter ZFS is all software.

You could download a 200MB FreeNAS Image, stick it on a USB drive, boot the USB drive in any PC, attach your ZFS disks and import the zpool and access your data in a matter of minutes.

Even if you don't export the zpool (in the case of a server failure) you can still force an import of the zpool on any new system.

 

[yashi]

To be true, the main problem I'm having right now is, i fell in love with those 1u/2u dell servers . Is there any way to make the noise level living-room acceptable? I mean, i know those things from work...they are like jets :/

Otherwise i will just stuff a bunch of hdds in my PC use the onboard raid and be done with it...Hopefully that solution will be low level enough that OS changes wont impact it/ notice it.

 

[drescherjm]

To be true, the main problem I'm having right now is, i fell in love with those 1u/2u dell servers

Snapraid will be better than these for storing media. I mean you should not use a raid server without a backup.

Is there any way to make the noise level living-room acceptable?

How about not putting your storage in your living room and go diskless on the machine that is in the living room.

 

[yashi]

Don't have a spare room. Besides that, i've read the powersupplies always take a huge amount of power/ fan always spinning even when offline etc. Is that true? Is power consumption that negligible for servers?

 

[drescherjm]

Don't have a spare room

Then I would look to 5XXX RPM drives and keep as few drives spinning as possible. Snapraid can help with this as well since only the disk with the file you need has to be spinning all other disks in your raid can spin down. For my snapraid setup I have the 2 parity disks as externals. They only need to be attached to my linux based HTPC when I want to sync the array or do other snapraid functions.

 

[yashi]

i don't really care about the HDD noise. I will get the cheapest ones, probably the Toshiba DT01ACA

 

[Dangman]

To be true, the main problem I'm having right now is, i fell in love with those 1u/2u dell servers . Is there any way to make the noise level living-room acceptable? I mean, i know those things from work...they are like jets :/

Otherwise i will just stuff a bunch of hdds in my PC use the onboard raid and be done with it...Hopefully that solution will be low level enough that OS changes wont impact it/ notice it.

Not a great solution. One of the problems that I also forgot to mention with onboard RAID controllers for parity RAID is the possibility of losing data should the PC suddenly lose power. Another problem is that large RAID arrays are rather stressful on both the system and the hard drives themselves when doing a RAID rebuild. As such, even after one hard drive has died, during the rebuild process, there's a possibility that another hard drive could die or the controller might accidentally drop another drive from the RAID array. You would lose all of your data at that point. Hence why RAID 6 is generally recommended for large RAID arrays of 6TB+ these days. Unfortunately, no consumer onboard RAID solution I'm aware of supports RAID 6.

While SnapRAID is a very solid solution, it's not quite an end-all be-all storage solution hands down. There are still plenty of other solid storage solutions out there. Personally, I highly recommend taking a second look at ZFS. These two articles are the main reasons why you should:
http://arstechnica.com/information-...-and-atomic-cows-inside-next-gen-filesystems/
http://arstechnica.com/information-...h-using-the-zfs-next-gen-filesystem-on-linux/

 

[FnordMan]

Don't have a spare room. Besides that, i've read the powersupplies always take a huge amount of power/ fan always spinning even when offline etc. Is that true? Is power consumption that negligible for servers?

Let's see here:
1: PSUs only take up the power they use + overhead for AC -> DC conversion.
2: fans only spin when it's on and in some cases a bit after shutdown to cool it off. Some PSUs offer a silent mode where at low loads it won't even turn the fan on
3: depends on your definition of "negligible" add enough drives and even low rpm drives will add up. Last I looked at it my fileserver was using ~160Watts at idle. (8x low rpm drives, 2x 7200rpm drives)

still: given all of that if your very careful you can still build a quiet server. One almost can't hear my desktop and the fileserver is only a bit noiser than that. (more fans: 3 up front, 2 in back counting the PSU)

 

[tazeat]

Don't have a spare room. Besides that, i've read the powersupplies always take a huge amount of power/ fan always spinning even when offline etc. Is that true? Is power consumption that negligible for servers?

Newer servers are MUCH MUCH better in the power department than older servers. I also wouldn't go with an off the shelf server if it's going to be in a living area unless its just a case that you build since the fans are usually meant for datacenters which can be a pain to replace and are usually very (sometimes very VERY) noisy. A cheap tower and a cheap low wattage i3+low end motherboard (onboard video with the new i3! No video card wasting power) can get you up and running with minimal power overhead, I think my 9 drive setup uses ~100-110W at idle. That said there will be people that say if you go raid, only go ECC memory etc etc. Pick a budget and draw out a hardware and software solution that will work for you. 2u/4u case vs full atx tower, Windows vs Linux vs Solaris, fancy hw raid cards vs sas expander setups, zfs vs mdadm vs windows raid vs a bazillian 3rd party solutions, etc

 

[TeeJayHoward]

still: given all of that if your very careful you can still build a quiet server.

This. The loudest part of my old fileserver was the hard drives themselves. 900RPM fans and an underutilized E3 in a quality chassis with a fanless PSU. You literally couldn't tell if it was on or not in the middle of the night unless you stuck your ear against the case. As for the power drain when the computer's turned off... I think I use about 3-4W of power for things like Wake-on-LAN, IPMI, etc. Disable all that, and it shouldn't use any.