RAID questions

[WhiteLighter]

Hi all,

I'm planning to build a media server and this is my first time dipping into something further than a basic pc. I'm planning to put my drives into RAID 5. My questions are:

1. if after a while I decided to add more drives, can I just plug in a new HDD and add it to the RAID array?
2. In case of RAID controller/MoBo failure, can I just replace them and plug in the drives without any data lost?
3. most of the file server setup that i've seen uses 1TB HDD, is there an issue with 2TB HDD in RAID?

Thanks ^_^

 

[drescherjm]

1. and 2. are yes for linux and zfs and all other non windows platform software raids.

3. I would not run more than 6 x 2TB disks in raid5. If you want more drives in a single array use raid6.

At work I am migrating several of my linux software raid 5 and raid6 arrays over to 2TB drives and in the process moving up to 2TB hitachi drives and using only raid6.

 

[Dangman]

1) Depends on what OS you're using and what controller card you're using. As drescherjm alluded to, generally, if you're using a non-Windows based software RAID, you can add a HDD to a RAID arry without losing data on the RAID array. However, if you're planning on using RAID with Windows, in that case, it's largely dependent on the controller you're using. Most onboard RAID controllers do not support Online Capacity Expansion, the ability to add a hard drive to a RAID array without losing data. Dedicated RAID controllers that have that feature tend to be a little expensive.

2) Yes, assuming that the RAID controller/mobo failure didn't corrupt the RAID array during the death. That is a possibility BTW.

3) 1TB drives are really cheap. And because the full compatibility list of 2TB drives and RAID isn't known exactly.

 

[WhiteLighter]

Thank you drescherjm and Danny Bui for all the info. So basically, in a Windows environment, a RAID set-up would only protect your data from HDD failures and not other hardware failures (controllers and Mobo). But on a linux environment, your data is protected further during hardware failures?

If my understanding (above) is true, then what would be the best way (and if I may add, cheap way) to protect large amount of data on a Windows environment?

On the Linux side, (I'm not really familiar with Linux) what would be a good version of Linux for a media server? and what build would be good for a low powered media server? I'm thinking of 3 simultaneous users accessing it for an HD video.

 

[Dangman]

Thank you drescherjm and Danny Bui for all the info. So basically, in a Windows environment, a RAID set-up would only protect your data from HDD failures and not other hardware failures (controllers and Mobo). But on a linux environment, your data is protected further during hardware failures?

Not really for the windows question. When it comes to Windows, you can in fact recover a RAID array when the mobo's onboard controller or a seperate controller dies. All you need is a storage controller that can read/use/recover the RAID array made by whatever controller you were using at the time. I.e

When it comes to Linux, you can pretty much move the RAID array to an entirely different motherboard or even Linux distro in the event of failure. So you can have a Linux software RAID array created in Ubuntu Server with an Intel motherboard, have that die, and then recover that RAID array in Fedora Linux with an AMD motherboard.

If my understanding (above) is true, then what would be the best way (and if I may add, cheap way) to protect large amount of data on a Windows environment?

Well my answer applies to both Linux and Windows: The best way to protect larges amount of data or absolutely critical data is simply having separate backups of that data. Or in other words, have another copy of that data stored on a separate system/media/device that's not connected to the main data source for long. I.e having that external hard drive connected to the PC at all times somewhat defeats the purpose of having an external hard drive for backup. Typical consumer backup solutions are another NAS, another file server, online storage, or even another PC. Whether or not those solutions are cheap is dependent on how much storage you really want safe at all costs.

RAID by itself is not backup. RAID just improves performance and provides uptime of your data in the event of a hard drive crash. But it is not backup or protect data very well as data in RAID array can still be accidentally and/or maliciously deleted, corrupted, modified, or accessed.

On the Linux side, (I'm not really familiar with Linux) what would be a good version of Linux for a media server? and what build would be good for a low powered media server? I'm thinking of 3 simultaneous users accessing it for an HD video.

Start off with Ubuntu Server IMO.

Hardware wise, answer these questions please:
1) What's your budget? Are tax and shipping included?
2) Where do you live?
8) When do you plan on building/buying the PC?

 

[WhiteLighter]

All you need is a storage controller that can read/use/recover the RAID array made by whatever controller you were using at the time. I.e

Thanks, not really familiar with it. Can you give an example (so that I can have a general idea on what to look for)

Hardware wise, answer these questions please:
1) What's your budget? Are tax and shipping included?
2) Where do you live?
8) When do you plan on building/buying the PC?

1) As low as possible ^_^ but decent enough. I'm not really sure since this will be my first time building one..If it's a gaming pc, I can say 1200 to 1500 since I know that I can build a good gaming pc with that budget. But for a file/media server, I actually have no idea ^_^ maybe 700-800..1200 if i'll be expanding the storage capacity.

2) I'm in California...bay area

3) I'll be moving out to a new place sometime around summer or fall this year, so I'm planning to build it after I move out.

 

[Dangman]

3) I'll be moving out to a new place sometime around summer or fall this year, so I'm planning to build it after I move out.

In that case, wait till you're actually moved in and ask again for file server hardware advice. A lot can change between now and summer or fall. As such, parts that we recommend today may not be good buys in the summer/fall. Hell even OSes that we recommend today may not be a good choice six to 8 months from now.

But if you were building a file server within a month, I would recommend this:
$120 - Intel Core i3-540 CPU
$280 - Supermicro X8SI6-F
$51- Kingston 2 x 2GB ECC Unbuffered DDR3 1333 RAM
$110 - Antec Truepower New TP-750 750W PSU
---
Total: $561 plus tax and shipping

The above setup would net you 14 hard drives right off the bat. That mobo is $280 but unlike many high-end consumer/gaming motherboards, you actually get a lot for the money as shown in this review of that motherboard:
http://www.servethehome.com/supermi...ew-including-onboard-lsi-sas-2008-controller/

 

[WhiteLighter]

Thanks Danny. I have skimmed through the review and it's really a big help ^_^ Any other suggestions are welcome. My late grandfather used to say "The accuracy of a decision is directly proportional to the adequacy of information"

 

[Dangman]

This is still a WIP but might as well throw it in anyway:
Amahi with Greyhole:
Links:
http://wiki.amahi.org/index.php/Greyhole
http://code.google.com/p/greyhole/
http://www.amahi.org/

Pros:
+ Lets you use different size drives and combine them into one drive pool ala WHS's Drive Extender feature.
+ Can set the number of copies of a folder/file that you want stored on other drives. So unlike WHS where only a single copy of that data is stored on a different drive, you set it to where that file/folder can be copied across every single hard drive in your PC. So 5 drives means 5 copies of your data
+ Supposedly you're still able to remove a drive from that Greyhole server and still be able to read the files on another PC without greyhole installed.
+ Free and works with Linux, which is also free

Cons:
- Still very new and experimental. So not all of the bugs may have been fixed or discovered yet.
- GUI is still relatively clunky
- Takes extensive amount of time to recover and repool data across a new drive after a drive failure

Notes:
* Should you delete a file while using Greyhole, all the other copies are deleted as well.
* While the above example recommend Amahi Home Server, you can use Greyhole in virtually any Linux based OS

Flexraid Basic:
Links:
http://en.wikipedia.org/wiki/FlexRAID
http://www.openegg.org/FlexRAID.curi

Pros:
+ Uses data based parity in which parity data is only kept of the file actually stored, not like with traditional RAID where parity data is is kept of the entire hard drives, regardless of whether or not you have data on those drives.
+ Allows use of different sized drives as a result of the data based parity setup
+ Free
+ Can be used with either Linux or Windows.

Cons:
- Not realtime parity so if the data needs to be protected at all times, not a good idea to use FlexRAID
- The FlexRAID needs periods of time of no usage at all in order to re-sync the RAID
- If multiple users are editing stored files, flexRAID is not recommended

Notes:
* FlexRAID Live does not have the cons of the FlexRAID Base. However it has not been released yet and even it is, it would still be a bit experimental at that stage.
* A little old but still good reading:
http://www.overclock.net/htpc/694948-has-anyone-tried-flexraid-their-media.html

Unraid:
Links
http://www.lime-technology.com/
http://lime-technology.com/wiki/index.php?title=UnRAID_Wiki[/QUOTE]

Pros:
+ Allows use of different sized drives.
+ Should one or multiple drives die, only the data on those dead drives are lost.
+ If the data on the hard drive isn’t being accessed, the hard drive is spun down until needed.

Cons:
- unRAID Free has a limit of 3 drives (2 data, 1 parity).
- unRAID Plus has a limit of 7 drives (5 data, 1 parity, 1 cache)
- unRAID Pro has a limit of 21 drives (19 data, 1 parity, 1 cache)
- Costs $120 for unRAID Pro (21 drives) and $69 for unRAID Plus (7 drives)
- Relatively low write speeds without that cache drive.

Notes:
* “unRAID™ is similar to RAID-4 in that for every n hard drives, there are n-1 data drives, and a single fixed parity drive” - From unRAID website

ZFS File Server:
Links:
ZFSguru NAS fileserver project
napp-it ZFS server appliance
Building your own ZFS fileserver

Pros:
+ Free
+ Fairly resilient against crashes, corruption, and such due to the copy-on-write feature as well as the snapshots feature.
+ Maintenance free in that the file system will periodically check and fix itself
+ Very flexible in terms of hardware and software changes.
+ Supports encryption of data

Cons:
- May be difficult to learn for a person who've never worked with BSD before
- Requires a large amount of RAM (4GB+ recommended)
- Cannot expand an existing RAID array with additional drives.

Notes:
* ZFS uses a different method of storage expansion:

What you cannot do, is expand an existing RAID-Z (RAID5) or RAID-Z2 (RAID6) array with one or more disks.
But, you can add new disks or RAIDs to an existing pool. So if you have a 4-disk RAID-Z, you can add another 4-disk RAID-Z so you have 8 disks. The second array would share free space with the first; in essence it would be a RAID0 of two RAID5 arrays. ZFS can expand this way.

What you can do, is expanding mirrors and RAID0's. In the example above that's what actually happened: a new array is RAID0-ed with the existing array. New created files will be written to both devices, for additional speed. Setting copies=2 would make files in that directory be stored on both RAID arrays; for extra redundancy.

* Here’s another thread filled with tips and an excellent hardware list for a large sized ZFS server:
http://hardforum.com/showthread.php?t=1572188