Storage Server Upgrade Advice Needed

[Cpmiller22]

+I need to overhaul my storage situation and am looking for some expert advice. +Right now I have a 64bit 2008 R2 server. +I have 2 500gb sata drives running raid 1 (managed by windows) a small IDE HD to run the os, and a third sata drive. +The raid drives are my prime storage for data files and music. +The third drive is for additional storage. +I'd like to upgrade to 2gb of redundant storage and then add 3-5TB for holding video content (I just got an HTPC). +I know I'm going to need to add a sata controller as my mobo only has 4 ports. +I'm looking for ideas on overhauling all of this. +I've been debating if I should stay on 2008, move to WHS, maybe even a Linux distro like freenas. +

Any suggestions would be much appreciated!

 

[Dangman]

What motherboard do you have now?
What's your max budget for this upgrade?
Did you really mean "2GB" or 2TB?

And just for the hell of it:
Amahi with Greyhole:
Links:
http://wiki.amahi.org/index.php/Greyhole
http://code.google.com/p/greyhole/
http://www.amahi.org/

Pros:
+ Lets you use different size drives and combine them into one drive pool ala WHS's Drive Extender feature.
+ Can set the number of copies of a folder/file that you want stored on other drives. So unlike WHS where only a single copy of that data is stored on a different drive, you set it to where that file/folder can be copied across every single hard drive in your PC. So 5 drives means 5 copies of your data
+ Supposedly you're still able to remove a drive from that Greyhole server and still be able to read the files on another PC without greyhole installed.
+ Free and works with Linux, which is also free

Cons:
- Still very new and experimental. So not all of the bugs may have been fixed or discovered yet.
- GUI is still relatively clunky
- Takes extensive amount of time to recover and repool data across a new drive after a drive failure

Notes:
* Should you delete a file while using Greyhole, all the other copies are deleted as well.
* While the above example recommend Amahi Home Server, you can use Greyhole in virtually any Linux based OS

Flexraid Basic:
Links:
http://en.wikipedia.org/wiki/FlexRAID
http://www.openegg.org/FlexRAID.curi

Pros:
+ Uses data based parity in which parity data is only kept of the file actually stored, not like with traditional RAID where parity data is is kept of the entire hard drives, regardless of whether or not you have data on those drives.
+ Allows use of different sized drives as a result of the data based parity setup
+ Free
+ Can be used with either Linux or Windows.

Cons:
- Not realtime parity so if the data needs to be protected at all times, not a good idea to use FlexRAID
- The FlexRAID needs periods of time of no usage at all in order to re-sync the RAID
- If multiple users are editing stored files, flexRAID is not recommended

Notes:
* FlexRAID Live does not have the cons of the FlexRAID Base. However it has not been released yet and even it is, it would still be a bit experimental at that stage.
* A little old but still good reading:
http://www.overclock.net/htpc/694948-has-anyone-tried-flexraid-their-media.html

Unraid:
Links
http://www.lime-technology.com/
http://lime-technology.com/wiki/index.php?title=UnRAID_Wiki[/QUOTE]

Pros:
+ Allows use of different sized drives.
+ Should one or multiple drives die, only the data on those dead drives are lost.
+ If the data on the hard drive isn’t being accessed, the hard drive is spun down until needed.

Cons:
- unRAID Free has a limit of 3 drives (2 data, 1 parity).
- unRAID Plus has a limit of 7 drives (5 data, 1 parity, 1 cache)
- unRAID Pro has a limit of 21 drives (19 data, 1 parity, 1 cache)
- Costs $120 for unRAID Pro (21 drives) and $69 for unRAID Plus (7 drives)
- Relatively low write speeds without that cache drive.

Notes:
* “unRAID™ is similar to RAID-4 in that for every n hard drives, there are n-1 data drives, and a single fixed parity drive” - From unRAID website

ZFS File Server:
Links:
ZFSguru NAS fileserver project
napp-it ZFS server appliance
Building your own ZFS fileserver

Pros:
+ Free
+ Fairly resilient against crashes, corruption, and such due to the copy-on-write feature as well as the snapshots feature.
+ Maintenance free in that the file system will periodically check and fix itself
+ Very flexible in terms of hardware and software changes.
+ Supports encryption of data

Cons:
- May be difficult to learn for a person who've never worked with BSD before
- Requires a large amount of RAM (4GB+ recommended)
- Cannot expand an existing RAID array with additional drives.

Notes:
* ZFS uses a different method of storage expansion:

What you cannot do, is expand an existing RAID-Z (RAID5) or RAID-Z2 (RAID6) array with one or more disks.
But, you can add new disks or RAIDs to an existing pool. So if you have a 4-disk RAID-Z, you can add another 4-disk RAID-Z so you have 8 disks. The second array would share free space with the first; in essence it would be a RAID0 of two RAID5 arrays. ZFS can expand this way.

What you can do, is expanding mirrors and RAID0's. In the example above that's what actually happened: a new array is RAID0-ed with the existing array. New created files will be written to both devices, for additional speed. Setting copies=2 would make files in that directory be stored on both RAID arrays; for extra redundancy.

* Here’s another thread filled with tips and an excellent hardware list for a large sized ZFS server:
http://hardforum.com/showthread.php?t=1572188

 

[rmd3003]

Just move to WHS v1. No RAID's to mess with. Works great, easy to setup and control.

 

[Cpmiller22]

Additional info:

My motherboard isn't anything fancy just a GF7100PVT-M3 that I got a good deal on a while back. The thing that I liked about it was the onboard gigabit ethernet and the DVI out. And you are correct, I did mean 2TB of redundant storage. My system has served me nicely for the last several months, but my data and music collection have just about filled up the 2 500GB mirrored drives. Additionally, I'm planning on moving my movies from optical storage (DVD's) to NAS like storage area that I can stream from my new Zatoc zbox id40 nettop.

Initially I was thinking I would just add 2 2TB drives to serve as the new redundant storage area, and then buy a 4 port internal sata controller where I would move the 2 500gb drives as well as add another 1-2 2TB drives and set it up as one big drive.

Now that I've been researching a bit more I've made a few key observations:
- Selecting a sata controller is much more involved that I thought it would be
- Win 2008 is not widely supported by most sata controllers
- I don't really use/need much other functionality of win 2008 other than as a file server/nas
- WHS sounds interesting, but I've read Drive Extender will be removed in the next version
- I've heard some people suggest staying away from hardware base raid as it can be very hard to easily recover if your raid controller crashes

All this considered, I'm still not certain which direction I should go. I think the first question is should I still move forward with 2 logical storage areas? The first would be highly redundant data. The second would be something that provides very little redundancy (i.e. if a single disk crashes I lose all the data on it.) Or should I consider one large redundant logical storage area.

I'd like to keep my system limited to no more than 8 disks leveraging the existing motherboard sata ports and 2x 500gb drives I have, but add a sata controller and some additional disks. So to refine my questions a bit more, I'd love some thought on the following:

- Recommendations on an inexpensive 4 port internal sata controller
- Recommendations on inexpensive 2TB drives
- Recommendations on which OS which best suit my requirements
- Recommendations on how I should structure my logical to storage to meet both my high redundancy data requirements as well as my requirements for a large logical media storage area.

Thanks in advance for your thoughts. I've been learning lots just reading through all the different posts in the forum.

-Chris

 

[Dangman]

Just to be clear: REDUNDANCY/RAID IS NOT BACKUP! All redundancy/RAID does is guarantee the uptime of your data in the event of a hard drive crash. It does not protect against malicious or accidental deletion, modification, corruption, or removal of data.

So if you want protection of your data, make sure that you have a copy of the critical data stored on a different medium or source that's not connected to the main source in any way.

So just keep that in mind.

- Recommendations on an inexpensive 4 port internal sata controller

Those do exist but they tend to be of very low quality and/or low performance. It's not until you get into the 8 port controller range where you can get the best bang for the buck combination of ports, speeds, and performance:
$100 - SuperMicro AOC-SAT2-MV8 PCI-X (PCI Compatible) 8 Port SATA Controller Card
$100 - SuperMicro AOC-SASLP-MV8 PCI-Ex4 8 Port SATA Controller Card
$120 - Supermicro AOC-USAS-L8i PCI-Ex8 8 Port SATA Controller Card (Will need some slight modification to work in normal ATX cases)

- Recommendations on inexpensive 2TB drives

Cheapest 2TB drive you can find will work. There's really no 2TB drive that you should definitely avoid.

- Recommendations on which OS which best suit my requirements

So far, your requirements makes WHS a good canidate for you due to the Drive Extender feature. However the problem is that WHS has issues with certain/many advance format 2TB drives which comprises a majority of 2TB drives out there.

- Recommendations on how I should structure my logical to storage to meet both my high redundancy data requirements as well as my requirements for a large logical media storage area.

That's largely dependent on your OS choice.

If you stick with Win 2K8, all you have to do is setup a normal RAID 1 array for your redundant data and then leave the rest of the drives as individual drives. Personally not a fan of the traditional JBOD setup where all drives are shown as one giant drive. The probelm that I've seen with those setups is that should one disk die, all that data is lost. However if you simply don't care about that, then go with a RAID 1 array + JBOD setup if using Win 2K8.

 

[Cpmiller22]

Thanks for the advice. I totally understand your point on redundancy vs backup. I forgot to mention in my current system I have an external 500GB drive that runs window backup once a week. This way if anything gets accidentally erased I have some time to recover.

Great pointers on the sata controllers. I noticed that the SAT2 controller isn't supported under server 2008. So if I wanted to stay on server 2008 the SASLP would be the best bet. I'm not familiar with SAS (been doing some reading on it since your post). A bit of a newbie question, but can I just connect the sata drives directly to the card? I didn't see 8 ports on the picture.

For the large media storage I think its much simpler for the HTPC if all the media is located in a single logical drive. Otherwise I have to manually manage what content is on each drive etc... I think I'd be ok with losing the data on the single disk if it were to die, but I don't want to lose an entire volume because of a single disk crash.

All things considered it sounds like WHS might be the best fit for me, but I’m concerned if they deprecate drive extender I’ll be stuck not being able to upgrade in the future. Also your comments on 2tb support issues are a bit concerning.

 

[Dangman]

Thanks for the advice. I totally understand your point on redundancy vs backup. I forgot to mention in my current system I have an external 500GB drive that runs window backup once a week. This way if anything gets accidentally erased I have some time to recover.

Gotcha

Great pointers on the sata controllers. I noticed that the SAT2 controller isn't supported under server 2008. So if I wanted to stay on server 2008 the SASLP would be the best bet. I'm not familiar with SAS (been doing some reading on it since your post). A bit of a newbie question, but can I just connect the sata drives directly to the card? I didn't see 8 ports on the picture.

Uhh, no. Actually if it worked with Win 2K3, it more than likely will work with Win 2K8. Oh and you need two of these cables in order to directly connect drives to a SAS controller:
$15 - 3ware SFF-8087 to Multi-lane SATA Forward Break-out Cable

All things considered it sounds like WHS might be the best fit for me, but I’m concerned if they deprecate drive extender I’ll be stuck not being able to upgrade in the future. Also your comments on 2tb support issues are a bit concerning.

Yeah, even if they did keep DE with the next iteration of WHS, it still would not be compatible with the this current iteration of DE as far as I can tell.

Just read through the list of options I provided above. One of those is bound to fit you just right.

 

[Cpmiller22]

Does anyone know if the saslp-mv8 or the sat2-mv8 will support true JBOD?

I've recently picked up 3 2tb 5900 seagate drives. I've decided to stay on Win2008. I'm going to run RAID 1 for 2 of the 2tb drives, and I'd like to have my 2 500gb drives and my other 2tb drive configured in true jbod so they are treated as one logical volume.

Thoughts/suggestions?