Help with a home server setup

[elde]

I've been reading [H]ardForum and especially Data Storage Systems subforum for a few months now, and it has been inspiring. Based on (and, again, inspired by) the threads here I moved a few of my hard drives from my desktop pc into my new htpc, replacing my old Antec Fusion with SilverStone SST-LC10B-E, which had more room for hard drives.

I was happy being able to access my files across my desktop and laptop, as well as being able to hook the hptc/home server into my tv and amplifier. Having all the hard drives in a htpc box under my tv just doesn't seem like a lasting idea, especially since they're not in any sort of raid at the moment, nor do I have backups made out of them (except for my photos).

Now, I'm looking into building my own home server, and storing it somewhere out of sight. I've been reading about different setups, and while I have a firm grasp on desktop hardware, servers and their internals, especially OS and software is somewhat an unknown territory for me. That is why I ask the experts for advice.

The home server I have in mind has enough 2TB or 3TB drives in a raid, low powered components to allow it to run 24/7 as well as being silent enough. I'm not sure what to do with the existing hardware I have, whether to keep using the htpc as is and buy separate server box, or use it's motherboard and processor (i3-540 + P7H55-M) in the server I had in mind.

I was wondering where to start with all of this. I'd like to read more about the home servers people have, and about the technology. I've been searching for resources to learn more about enclosures, server board properties that make them different from the desktop/laptop ones and about the operating systems. Is normal Windows 7 enough, or should I go with something heavier? WHS? SBS? Server 2008R2? I'm interested in ZFS too, but my expertise in unix systems goes back to running a linux on a 386SX in early 90's as a young kid. Also, I've been wondering about the RAID setups - are the chipset-based RAIDs enough, or should I invest in a hardware RAID controller? I hear good things about IBM M1015, but it might be a bit too heavyweight for my solution.

If you have some good starter points I could look into or could share your home server setups and reasons why you got the parts you did, I'd be very grateful.

 

[Dangman]

I'm not sure what to do with the existing hardware I have, whether to keep using the htpc as is and buy separate server box, or use it's motherboard and processor (i3-540 + P7H55-M) in the server I had in mind.

I'd recommend keeping the HTPC to keep things simple. Do you really want to build/rebuild/work on/troubleshoot two different PCs at the same time? Keeping the HTPC as is with its current parts will make sure that you only have one machine to build i.e the server.

I was wondering where to start with all of this. I'd like to read more about the home servers people have, and about the technology. I've been searching for resources to learn more about enclosures, server board properties that make them different from the desktop/laptop ones and about the operating systems.

Well you want to see what other people have for their home servers, hit up this thread:
http://hardforum.com/showthread.php?t=1393939

If you want to know more about home servers in general, this site is a good start
http://www.servethehome.com/

Also this might help you a tad:
Amahi with Greyhole:
Links:
http://wiki.amahi.org/index.php/Greyhole
http://code.google.com/p/greyhole/
http://www.amahi.org/

Pros:
+ Lets you use different size drives and combine them into one drive pool ala WHS's Drive Extender feature.
+ Can set the number of copies of a folder/file that you want stored on other drives. So unlike WHS where only a single copy of that data is stored on a different drive, you set it to where that file/folder can be copied across every single hard drive in your PC. So 5 drives means 5 copies of your data
+ Supposedly you're still able to remove a drive from that Greyhole server and still be able to read the files on another PC without greyhole installed.
+ Free and works with Linux, which is also free

Cons:
- Still very new and experimental. So not all of the bugs may have been fixed or discovered yet.
- GUI is still relatively clunky
- Takes extensive amount of time to recover and repool data across a new drive after a drive failure
- Amahi has a tendency to try to become the default DHCP and DNS cache server for the entire network
- Uses the old “landing drive” system of the older iteration of WHS

Notes:
* Should you delete a file while using Greyhole, all the other copies are deleted as well.
* While the above example recommend Amahi Home Server, you can use Greyhole in virtually any Linux based OS

Flexraid Basic:
Links:
http://en.wikipedia.org/wiki/FlexRAID
http://www.openegg.org/

Pros:
+ Uses data based parity in which parity data is only kept of the file actually stored, not like with traditional RAID where parity data is is kept of the entire hard drives, regardless of whether or not you have data on those drives.
+ Allows use of different sized drives as a result of the data based parity setup
+ Free
+ Can be used with either Linux or Windows.

Cons:
- Not realtime parity so if the data needs to be protected at all times, not a good idea to use FlexRAID
- The FlexRAID needs periods of time of no usage at all in order to re-sync the RAID
- If multiple users are editing stored files, flexRAID is not recommended

Notes:
* FlexRAID Live does not have the cons of the FlexRAID Base. However it has not been released yet and even it is, it would still be a bit experimental at that stage.
* A little old but still good reading:
http://www.overclock.net/htpc/694948-has-anyone-tried-flexraid-their-media.html

Unraid:
Links
http://www.lime-technology.com/
http://lime-technology.com/wiki/index.php?title=UnRAID_Wiki[/QUOTE]

Pros:
+ Allows use of different sized drives.
+ Should one or multiple drives die, only the data on those dead drives are lost.
+ If the data on the hard drive isn’t being accessed, the hard drive is spun down until needed.

Cons:
- unRAID Free has a limit of 3 drives (2 data, 1 parity).
- unRAID Plus has a limit of 7 drives (5 data, 1 parity, 1 cache)
- unRAID Pro has a limit of 21 drives (19 data, 1 parity, 1 cache)
- Costs $120 for unRAID Pro (21 drives) and $69 for unRAID Plus (7 drives)
- Relatively low write speeds without that cache drive.

Notes:
* “unRAID™ is similar to RAID-4 in that for every n hard drives, there are n-1 data drives, and a single fixed parity drive” - From unRAID website

ZFS File Server:
Links:
napp-it ZFS server appliance
Building your own ZFS fileserver
ZFSguru NAS fileserver project

Pros:
+ Free
+ Fairly resilient against crashes, corruption, and such due to the copy-on-write feature as well as the snapshots feature.
+ Maintenance free in that the file system will periodically check and fix itself
+ Very flexible in terms of hardware and software changes.
+ Supports encryption of data

Cons:
- May be difficult to learn for a person who've never worked with BSD before
- Requires a large amount of RAM (4GB+ recommended)
- Cannot expand an existing RAID array with additional drives.

Notes:
* ZFS uses a different method of storage expansion:

What you cannot do, is expand an existing RAID-Z (RAID5) or RAID-Z2 (RAID6) array with one or more disks.
But, you can add new disks or RAIDs to an existing pool. So if you have a 4-disk RAID-Z, you can add another 4-disk RAID-Z so you have 8 disks. The second array would share free space with the first; in essence it would be a RAID0 of two RAID5 arrays. ZFS can expand this way.

What you can do, is expanding mirrors and RAID0's. In the example above that's what actually happened: a new array is RAID0-ed with the existing array. New created files will be written to both devices, for additional speed. Setting copies=2 would make files in that directory be stored on both RAID arrays; for extra redundancy.

* Here’s another thread filled with tips and an excellent hardware list for a large sized ZFS server:
http://hardforum.com/showthread.php?t=1572188

Linux MDADM Software RAID:
Links:
How to create RAID using Ubuntu Software RAID
Linux Software RAID
MDADM
The Software-RAID HOWTO

Is normal Windows 7 enough, or should I go with something heavier? WHS? SBS? Server 2008R2? I'm interested in ZFS too, but my expertise in unix systems goes back to running a linux on a 386SX in early 90's as a young kid.

Depends on your exact needs for the OS of your server. I.e familarity, features, performance, hardware support, software support, etc, etc

Also, I've been wondering about the RAID setups - are the chipset-based RAIDs enough, or should I invest in a hardware RAID controller? I hear good things about IBM M1015, but it might be a bit too heavyweight for my solution.

Well first and foremost, keep this in your mind: RAID IS NOT BACKUP! All it does is guarantee uptime of your data in the event of hard drive failure. It does not protect against accidental or malicious deletion, corruption, modification, or locking of your data.

Anyway, you need to figure out your OS choice first before you can start talking about RAID and such. Your choice of OS will determine what hardware you'll need for RAID. For example, if you're using Linux, you don't need to buy a true hardware RAID card. All Linux software RAID needs are SATA ports. If you're using Windows and want high-performance, you basically need the true hardware RAID. With that said, I would not bother with using the motherboard's onboard RAID unless you absolutely need Windows and absolutely need RAID and don't care about performance whatsoever.

 

[aviat72]

Danny:
Thanks for the detailed answers. I have a similar thread open and have questions about ZFS

Also I am curious about your comments about the motherboard onboard RAID. With the kind of processing capability current gen CPUs have, the non-parity type raids (RAID0, RAID1, RAID10), should not add any significant overhead. Where does the performance bottleneck come from?

 

[Dangman]

Also I am curious about your comments about the motherboard onboard RAID. With the kind of processing capability current gen CPUs have, the non-parity type raids (RAID0, RAID1, RAID10), should not add any significant overhead. Where does the performance bottleneck come from?

If you're asking in regards to the parity type raids, the performance bottleneck is probably due to the design and implementation of the firmware and drivers as well as the choice of OS (Windows). In other words, whereas Linux software RAID and ZFS RAID is handled by the OS itself, with fakeRAID, the controller is pretending to be a raid controller in the BIOS but once the OS boots, then the parity calculations are then handled via a driver running in the background of the OS.

As an example, the ICH10R RAID controller can supposedly provide a RAID 5 array with write speeds of up to 70MB/s if writeback is enabeld IIRC while using Windows and the required driver. However on the same exact hardware but not using the ICH10R or its related drivers to start and maintain the arrays and using Linux or whatever ZFS supported OS, you can easily get speeds 4 to 6 times that. Considering that both methods uses the PC's CPU to handle the parity calculations, the difference in performance is more than likely due to the implementation of those methods.

 

[elde]

Wow, just wow.

I posted my questions here to get a few links to sites explaining a few terms or example server configurations, but got a wealth of information in return.

It'll take some time to chew through all that, but in the meantime, my hat's off to you, Danny_Bui. Thank you.