New FileServer Build 20TB+

[sebjavierw]

Hi
I have am about to exceed my current storage capacity (HP MicroServer 8TB)

I have decided on buying a Norco RPC-4224 (24 Bay Chassis)
I am a little confused about SAS expanders, RAID Cards etc..

Basically I will have about 10TB to go into the chassis at the beginning.
WD 3TB Red Drives
Can I then add more later into the RAID array without losing data?

I am also looking at a motherboard, PSU and other peripherals to go with the chassis.
If people could recommend these that would be great. Bear in mind I would like them to be able to support 24 HDDs rather than having to change motherboard later down the track when expanding.

I am looking to keep this build as cheap as possible.
I am not sure whether RAID cards or the motherboard would be better for holding the array.
Note: I think RAID 60 would be the best option but if anyone has better ideas please comment.

Thanks in advance

 

[Dangman]

Basically I will have about 10TB to go into the chassis at the beginning.
WD 3TB Red Drives

10TB RAW or 10TB after RAID?

Can I then add more later into the RAID array without losing data?

If you're going with a traditional RAID setup, generally yes unless you have really bad luck.

I am also looking at a motherboard, PSU and other peripherals to go with the chassis.
If people could recommend these that would be great. Bear in mind I would like them to be able to support 24 HDDs rather than having to change motherboard later down the track when expanding.

What country are you in?

I am looking to keep this build as cheap as possible.
I am not sure whether RAID cards or the motherboard would be better for holding the array.

If you want cheap, then you should be avoiding Windows based solutions and true hardware RAID cards. That means you're basically looking at software RAID setups with relatively cheap HBAs and Linux or BSD based OS that supports MDADM RAID and ZFS respectively.

Note: I think RAID 60 would be the best option but if anyone has better ideas please comment.

Look into ZFS:
http://arstechnica.com/information-...-and-atomic-cows-inside-next-gen-filesystems/
http://arstechnica.com/information-...h-using-the-zfs-next-gen-filesystem-on-linux/

 

[sebjavierw]

Either doesn't bother me at this stage. My plan is to fill the chassis with drives so 3TBx24 = 61.44TB RAW or 57.22TB RAID. I can't fill it straight away so over time this will be done.

Australia

While I want it to be cheap I also want it to be reliable, safe and expandable. If that involves spending a bit more then so be it.
Thankyou for the link I will have a closer look

 

[Dangman]

Assuming that you're going the software RAID route, I recommend something like this:
$120 - Intel Core i3-4130 CPU
$250 - Supermicro MBD-X10SL7-F-O Intel C222 mATX Motherboard
$94 - Kingston KVR16E11/8 8GB DDR3 1600 ECC Unbuffered RAM
$140 - SeaSonic SS-750KM3 750W Modular PSU
$225 - LSI 9211-8i SATA Controller Card
$36 - 2 x 3ware CBL-SFF8087OCF-06M SFF-8087 to Multi-lane SATA Forward Break-out Cable
----
Total: $865 plus tax and shipping

That motherboard has 14 SATA ports and that controller will add another 8 SATA ports. Yes you would still be short two SATA ports. ECC RAM is apparently needed for ZFS for added security of the data.

 

[sebjavierw]

Would you recommend software RAID or hardware RAID keeping in mind what the total storage is going to be in the future 24 drives (3/4TB each).
Would software RAID support RAID 60 or the equivalent?

For the sake of planning how would I get around the short 2 SATA port issue?

Thanks for the help

 

[Dangman]

Would you recommend software RAID or hardware RAID keeping in mind what the total storage is going to be in the future 24 drives (3/4TB each).

Software RAID. ZFS' advantages are too great to ignore. To accomplish what you want with hardware RAID, it's going to cost quite a lot more.

Would software RAID support RAID 60 or the equivalent?

Yes.

For the sake of planning how would I get around the short 2 SATA port issue?

Get the cheapest 2 SATA port controller you can find. Most software RAID solutions lets you use drives connected to different controllers as part of one RAID array.

Though a more...expansive way would be to just a HP SAS expander and hook it up to the LSI card I linked via a SFF-8087 to SFF-8087 cable:
http://hardforum.com/showthread.php?t=1484614

That way, you'll get 14 SATA ports from the motherboard, 32 SATA ports from the expander, and 4 SATA ports from the LSI card for a total of 50 SATA drives.

I also recommend hitting up this site for a bit more info:
http://www.servethehome.com/category/client-tips-and-tricks/the-big-whs-30-drive-whs/

 

[sebjavierw]

Ok thanks so much for your help I will start finding costs for Australia and work out everything.
Do you mind if i PM you if I need some more assistance?
Thanks

 

[Dangman]

Ok thanks so much for your help I will start finding costs for Australia and work out everything.
Do you mind if i PM you if I need some more assistance?
Thanks

Just post the questions here. There are times where I could be wrong. With a project of this cost and size, you should be getting as many opinions and information as possible.

 

[MrGuvernment]

ZFS but on FreeBSD where it is meant to be.

do not use intel raid or raid 5 or raid 6 with built in raid, you WILL lose data on a rebuild period.

 

[TeeJayHoward]

Assuming that you're going the software RAID route, I recommend something like this:

Good combo. I'm running something very similar myself. If you're looking to reduce costs further, though... Reverse breakout cable + that motherboard (X10SL7) + a chassis which has an SFF-8087 port (Like the RPC-4224) totally eliminates the need for a controller. I've not done it myself (Can't find a reverse breakout cable locally to test it!) but logically, it would work. A forward breakout cable wouldn't be required in any situation - You'd need a 8087-to-8087 from that card. It's designed to connect to an SAS expander. Don't think I've ever seen a 8087 backplane that ISN'T an expander. (edit: See below. They DO exist!) So if you got the card, you could have 256 drives connected to that one card. The onboard ports wouldn't be required at all. In fact, you could probably save money and get the X10SLM, X10SLH, X10SLX, or X10SLL motherboard instead if you're going to use a dedicated controller. Whatever the cheapest X10___-F one is.

Edit: Nevermind. Who the hell designs a system with 6 freaking backplanes? That explains why you recommended the X10SL7... Three reverse breakout cables, the controller, and half of a reverse breakout cable... All for just 22 drives. Jesus... Is a used SuperMicro 846 available anywhere near you?

Price comparison:
Supermicro 846 on eBay: ~$820
Single reverse breakout cable: $30
============================
$850

Norco RPC-4224: $430
Controller: $200
Reverse breakout cable x3: $60
8087-to-8087 cable x2: $60
PSU: $100
============================
$850

 

[dexvx]

Any server chassis with a proper SAS backplane is going to be expensive (around 1K new).

If this is just for personal use, I'd just opt for the multiple backplanes like the ones on the Norco. If you want a cheaper option, you can just use SFF-8087/SATA breakout for the Rosewill chassis. I got the L4500 because I'm never going to hotswap.

Edit: Not all Supermicro 846's have a single SAS backplane. The cheaper ones use 6x SFF-8087 connectors.

 

[TeeJayHoward]

Edit: Not all Supermicro 846's have a single SAS backplane. The cheaper ones use 6x SFF-8087 connectors.

Incorrect. There are 6x SFF-8087 connectors on a single backplane on the E2 models. Two in, four out. That's their most expensive chassis, not their cheapest. It allows for dual-porting to be used.

 

[SirMaster]

Wouldn't it be cheapest to go with the Norco 4224 and then stick with the X10SL7 and then just get 2 more LSI SAS cards like this?

http://www.ebay.com/itm/LSI-Interna...sk_Controllers_RAID_Cards&hash=item3cdf9a7cf9

That would be $250 for the mobo, and $175 for 2 of these LSI cards. Then use 2 reverse breakout cables from the motherboard and the 4 SAS provided by the 2 SLI cards to get the 6 SAS connections you need for the cheap 4224 case.