• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Need help! 30TB+ storage.

C

cokenbeer

n00b
Joined
Sep 20, 2011
Messages
1
I'm quickly outgrowing consumer NAS systems and am looking to build a storage server. My home network includes 5 PC's with over 10TB of data between them. The rest of my data is stored on a QNAP NAS which is quickly reaching its limits.

I'd like to create a flexible storage system with at least 30TB and the ability to easily add more storage as required. The sever will also be used to stream media and run ESXi.

I have the following components from some old builds and can use them here if they're suitable. If not I'm open for any suggestions.

Intel i7 980x
Intel BOXDX58SO2 -> http://www.newegg.com/Product/Product.aspx?Item=N82E16813121470
6GB Corsair DDR3

I've also looked through a couple different chassis:

http://www.supermicro.com/products/chassis/4U/846/SC846E2-R900.cfm
Seems to be a good fit?

From here it gets really confusing. I guess depending on whether I go with ZFS or a hardware RAID card will make a big difference in what I buy. I'm not entirely clear on the advantages of going one way or another.

I'm also not sure which HDD's would work best. I'd like to go with 2TB drives minimum, but if some of the lower cost 3TB drives are acceptable then I'll go with those instead.

The OS will be installed on a SSD.

EDIT: Forgot to add, the budget is flexible. I'm looking for performance/dollar.

That's where I'm at so far, looking for some advice and guidance before I place the order. Once things start coming together I'll post some pics.

Thanks!
 
cokenbeer said:
I'd like to create a flexible storage system with at least 30TB and the ability to easily add more storage as required. The sever will also be used to stream media and run ESXi.

http://www.supermicro.com/products/chassis/4U/846/SC846E2-R900.cfm
Seems to be a good fit?

From here it gets really confusing. I guess depending on whether I go with ZFS or a hardware RAID card will make a big difference in what I buy. I'm not entirely clear on the advantages of going one way or another.

I'm also not sure which HDD's would work best. I'd like to go with 2TB drives minimum, but if some of the lower cost 3TB drives are acceptable then I'll go with those instead.

The OS will be installed on a SSD.

Thanks!
Click to expand...

i would
- use a case with a LSI SAS2 expander like
http://www.supermicro.com/products/chassis/4U/846/SC846E16-R1200.cfm

optionally:
use a cheaper 16 bay case and buy 3 TB disks
but use always the newer SAS2 expanders from LSI

- add a LSI SAS HBA like a LSI 9211-8i
-> best for expander use and supports disks > 3 TB
(one SAS port for the expander, the other to an external bracket for further expansion)

- for all-in-one based on ESXi, you should think about 16 to 24GB RAM
(use as much as possible for SAN/NAS VM (used as Cache) if thats the primary use)

- If you want to virtualize other systems than the SAN OS itself, think about
two pools, one for the filer and backup and a smaller and faster for the VM's

- add a SSD read cache for the large pool

- use only 512b disks (ex Hitachi) not the 4 k ones

- think about backups (your qnap?)

!! and: go the ZFS way without Hardware Raid
 
Some tips from me:

1) Use an LSI SAS card. Firmware is rock solid and most can be flashed as either a RAID card, or a straight HBA (IT mode). For most installs, if you're using a SAS expander as well, use an LSI 9211-8i coupled with an HP SAS expander. The HP SAS expander can be connected using dual uplinks, doubling bandwidth to the expander.

2) Don't cheap out on the case. I opted to do so and go the route of the Norco RPC 4224. Whiles an okay case, I have regretted not going for something more solid, like the supermicro cases. When you combine PSU + Case + SAS Backplane, you end up paying around the same anyway.

3) Use ZFS if possible and if you're comfortable. It'll future proof things for you and is well supported by the community. End to end checksumming, awesome snapshot capabilities, snapshot sending, diverse sharing capabilities (NFS, CIFS, iSCSI, FC), simple expansion; can't beat it.

4) RAID6/RAIDZ2 - disks are cheap. Rebuild time on a RAID array grows as the size of your disks grow. The likelihood of you loosing a disk is greatest when your array is rebuilding after a previous failure. Save yourself the grief of loosing _all_ your data, and have double parity.

5) Be aware of 4K sector disks. Note I say be aware and not beware? 4K is going to be the way of the future, however you need to make sure your OS happily works with these. ZFS works fine, though I highly recommend looking into formatting your pool using ashift=12 vs ashift=9. Windows also supports 4K.

Finally) No matter what you use, you still need to be able to back up the important stuff. I'm not saying backup your DVD collection or anything like that, but still be able to back up your family photos, documents, etc. Re-purposing your existing NAS device for this purpose is a good idea. Just make sure the backups are scheduled and automatic (cron job, or windows scheduled task). Also, monitor your backups to ensure they're taking place.


Hardware suggestions:
1) Motherboard: Supermicro
2) Memory: Kingston, ECC RDIMM's
3) HBA: LSI SAS cards
4) Expander: HP SAS Expanders
5) Hard disks: Hitachi, Samsung, WD/Seagate (order of preference)


My slightly more that 2 cents :)
 
You must log in or register to reply here.
Back
Top