• A Great friend to the HardForum with a great kid that he is trying to get a scholorship to continue his schooling. Please give hime a vote! Only 24 hours left! Thanks.
    If you have an VOTE FOR KEENAN!

Designing first Storage Pool

moyam01

n00b
Joined
Jul 16, 2011
Messages
6
Hello,

I am in the process of designing my first storage pool.

Goals:

Like implied above, I want it to act as a storage pool for multiple machines. First, I would like it act as the main hard drive for my Desktop machine.

Ex. The desktop machines would contain a small hard drive that contain the OS and all of the boot stuff. But directories that are designed to hold all my files would be on the storage pool, for example my /home/user/Documents or Videos or whatever else will lead to the storage pool. I contemplated having the whole home folder on the pool, but I don't know what implications it will have on other machines when it comes to the configuration files that are stored in the home directory. This setup will allow me to access the files on other systems as well, as I plan to mount the correct folders on startup. This will also serve as a space for my virtualbox VM's.

Connected to the server is also a media server (might be virtualized). Think along the lines of a server for boxee box. Might also be used as a file server.

The last is my laptop. Here things become a little more complicated. I want the laptop to have access to the files when I am home and on the network (think My Videos in Ubuntu is mounted to the server), but when I am away the system is still capable of functioning if the network isn't present. When I get home, the files in the laptop will be automatically uploaded to the current file collection.

Last but most important is the backup. I will need versioning backup, but only for certain files. (like documents and other files I specify). I would like for this to be done automatically.

I believe ~6TB is sufficient for me to start with but the system must have enough room for me to expand if necessary.

Constraints:

Price:

While I have no direct price figure yet, anything over $1500 is quite a lot, and might cause me to re-think this whole thing.

And that's it.

What I am looking for is what software should be used on the main backup pool and how would you handle the integration of the storage pool into the operating system. Also, hardware advice is welcome.

Another big topic is RAID. I have no idea what to choose. I know exactly what it is and when to use it, but question if it a cost-effective idea for a home environment, specially when i plan to have consistent backups.

Thanks for any assistance,
moyam01
 
Basically, if your network storage is relied upon by everything, you want it to be redundant.

LSI SAS2008 controller, RAID 1 or 10 and 4x 3TB drives. Either get a Xeon E3 motherboard with the LSI controller onboard (e.g. Tyan S5512WGM2NR) or get an add-in card. That has you ready for WHS 2011. If you decide on Linux or ZFS solutions, you can skip the controller/ get an IBM M1015. Add in ECC memory and a decent PSU and you are done. Easily <$1000. See site to get you started. I would advise against using a consumer board and just get something with KVM-over-IP and Intel NICs to save you trouble.
 
I am planning on going Linux for the back end. Are you suggesting that I go software raid if I go with linux?
 
Are you suggesting that I go software raid if I go with linux?

I would stick with software raid on most non-windows systems. Remember that the argument about needing a hardware raid card because of high cpu usage is long over for non windows software raid.

For example at work I have a core2quad linux system with 9 hitachi 2TB 7k2000s in raid 6 using linux software raid 6. I use a combination of the intel motherboard ports and an Intel SASUCI8 to connect the 9 drives. For this array I am seeing 700+ MB/s reads and 500+ MB/s writes. That is with less than 12% CPU usage on a single core of the quadcore. Rebuilds will have significantly higher cpu usage (but still manageable). Rebuilding is multi-threaded on modern linux kernels (I believe 2.6.35+). Anyways the rebuild (after replacing a drive) on this 14TB software raid 6 array takes about 8.5 hours for the entire array.
 
Last edited:
I would stick with software raid on most non-windows systems. Remember that the argument about needing a hardware raid card because of high cpu usage is long over for non windows software raid.

For example at work I have a core2quad linux system with 9 hitachi 2TB 7k2000s in raid 6 using linux software raid 6. I use a combination of the intel motherboard ports and an Intel SASUCI8 to connect the 9 drives. For this array I am seeing 700+ MB/s reads and 500+ MB/s writes. That is with less than 12% CPU usage on a single core of the quadcore. Rebuilds will have significantly higher cpu usage (but still manageable). Rebuilding is multi-threaded on modern linux kernels (I believe 2.6.35+). Anyways the rebuild (after replacing a drive) on this 14TB software raid 6 array takes about 8.5 hours for the entire array.

What about in WHS2011, would you recommend software raid? maybe something like flexraid? Im planning a similar build as you posted, but on WHS2011, with 8-10 drives, but with no extender on WHS2011, i was thinking on going raid.
 
Ok. So software RAID. Since I'm doing that, I assume the controller does not matter, but processor and memory does. I assume no problems with Intel SASUC8I if it comes to it.

However, is there anything like this that has compatibility for SATA 3? I probably wont need it as the drives I will be running probably won't touch the sata 2 limits, but thinkinga little further ahead into the future.

Let's talk about the software. Specifically, what distro should I run on the box to handle the raid. I am most familiar with Debian-style systems, but I can learn anything else if necessary. So what do you recommend?

Also, continuing with the hardware theme, what would be a good motherboard now that I decided to run software raid. Does it matter at all?

Oh, and what about hot-swap cases. I was looking at one of the Norco units, but noticed that there were many mixed reviews on the product.

Thanks,
moyam01

EDIT: Take a look at this motherboard if you can and let me know if its ok. r if you have a better suggestion let me know. http://www.newegg.com/Product/Product.aspx?Item=N82E16813182211

Foudn the newer version: http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253
 
Last edited:
You sound like you are in the same position I was in a couple of weeks ago. Here's my story. Right now, I did away with ESXi and went with a linux box. I then installed "ZFS on Linux" (currently RC5) and a M1015 controller converted to a LSI 9210-8i via firmware change (required another computer to successfully flash). You could also use OpenIndiana for a more stable version of ZFS. It has lots of power and room for growth.
 
However, is there anything like this that has compatibility for SATA 3? I probably wont need it as the drives I will be running probably won't touch the sata 2 limits, but thinkinga little further ahead into the future.

I would bet that will be a long time in the future (2020?). Look how many years for SATA hard drives to finally break the 150 MB/s limit of SATA1. With that said I believe (from reports on this forum - i do not have the card) the M1015 that you can get cheap on eBay is a good choice if you can reflash the firmware
 
Ok then, but it brings up another question:

What is the difference between the two controllers if I am using software raid. It seems to me the IBM board is only beneficial if you are in fact running a hardware raid, and you are monitoring it through the card as it has many monitoring information.

Also, what are the benefits on changing the firmware if again we are talking about software raid.

I think I might be missing something. I see on ebay the IBM cards can be hadd for about the same price as a new Intel controller. Is it worth it?

If there are any other controllers out there which are worth looking at, please let me know. In the meantime, I will be do all the research I can.

Thanks again,
moyam01
 
Natively, the M1015 does RAID 0,1,and 10. It is designed to do Raid. Changing the firmware makes it into a HBA, meaning a controller passes untouched drives to the OS. This allows software raid (unraid, and zfs included) more direct access to the actual drive. I also believe it also enables use expanders, although I've seen some warnings about sata on expanders.

Sata speeds can be useful with not only SSDs, but if you use sata port multipliers or SAS expanders, which will share the bandwidth. Using an external enclosure connected via either esata multiplier or USB 3.0 and a "zfs send" will be my backup solution. I don't know however if either my onboard ich10 or the M1015 it will support a multiplier.
 
Last edited:
Also, continuing with the hardware theme, what would be a good motherboard now that I decided to run software raid. Does it matter at all?

....
Foudn the newer version: http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253
That's a good motherboard choice.

Oh, and what about hot-swap cases. I was looking at one of the Norco units, but noticed that there were many mixed reviews on the product.
The Norco cases are really good cases for the money. May not be the best and may have some flaws but for the price you're paying, really can't complain. I recommend either of these cases:
$350 - NORCO RPC-4220 4U Rackmount Server Case
$400 - NORCO RPC-4224 4U Rackmount Server Case

If you want, you can connect four of the onboard SATA ports on your motherboard to either of the Norco cases with this cable:
$15 - Norco C-SFF8087-4S SFF-8087 to Multi-lane SATA Reverse Break-out Cable

To connect the Intel SASUC8I or IBM M1015 or LSI 9210-8i to the case, you need two of these cables per how many cards you plan on having in the system:
$15 - 3ware SFF-8087 to Multi-lane SATA Forward Break-out Cable

So if you go with two cards, that gives you 16 HDD capability right there.

Sata speeds can be useful with not only SSDs, but if you use sata port multipliers or SAS expanders, which will share the bandwidth. Using an external enclosure connected via either esata multiplier or USB 3.0 and a "zfs send" will be my backup solution. I don't know however if either my onboard ich10 or the M1015 it will support a multiplier.

ICH10R does support port mulpliers (Under the "Choose Intel Matrix Storage Technology" section):
http://www.intel.com/design/chipsets/matrixstorage_sb.htm
 
Back
Top