DIY NAS

[jfromeo]

Hello all.

Another DIY NAS builder here. I am looking for a future-proof rig, based on modularity and expandability. That is why I have decided to build an own NAS rig from scratch instead of buying a comercial solution.

Characteristic of the rig:

Main use: Homeserver for streaming music (flac), videos (mkv) and games (7z)
Budget: ~700-800€ (HDDs not included)
Hard disks: 8xHitachi 7K4000 4TB
RAID: 5 or 6 (not decided yet)
SO: FreeNAS 9.1 x64

Main specifications of the system:

CPU:Intel Core i3-3220T (~115€)
MB: Asrock H77M-I (~90€)
RAM: 2x4GB Samsung DDR3-1600 CL9 Low Voltage (~60€)
SSD: Crucial M4 64GB (~75€)
HDD: 8xHitachi 7K4000 4TB
PSU: Seasonic SS-300SFD (40€)
FANS: 2xGentle Typhoon 120mm@800rpm (~25€)
RAID: ? (~150€?)
CASE: SilverStone DS380 (not released yet) (~200€?)

An image of the case from SilverStone to be released:

I have several questions about some components.

Regarding the RAID card, I need an 8 internal SATA3 PCI-e card with RAID 5 and 6 capabilities. I have done some research and there are some cheap cards for 100€-150€ with 8 internal ports and RAID 5 capability, but for those with RAID 6, the price goes all up to 400-500€ (checked RocketRAID, 3Ware, LSI, Adaptec, Areca...). Why is this feature so pricey? Which one would you recommend for a home DIY NAS?

Regarding the RAM, does the low voltage sticks (1.35v) make a difference in power consumption with usual voltage rated ones (1.5v)? Is 2x4GB overkill for a NAS, or worth given the little difference in price with a 2x2GB set?

And finally, concerning the SSD, is 64gb enough for running FreeNAS with this setup?

Feel free to change any component

Thanks a lot in advance.

 

[Proplus]

raid 5 with 8 large drives is not a good idea. If you lose a drive it may take days to rebuild, and if you lose another drive in that time, you're hosed.

raid 6 is safer in that regard, but with sata drives still a concern. We do however use raid 6 arrays with 8 sas drives in our storage (with GPFS data mirroring and failover groups on top) for the high performance nas solutions my company sells.

Most people here are going to say "use ZFS". There's plenty of information about it in this forum!

 

[jfromeo]

Thanks proplus!

So, if I have understood it well, ZFS is some sort of filesystem that also allows "RAID" configurations. So I could format the pool with the equivalent to R5 (RAIDZ) or R6 (RAIDZ2) without the need of a dedicated RAID card (the "real" RAID controller cards that do RAID via hardware are at a ~500€ price tag, while the software based ones are at ~150€).

So, with a software-based RAID card (for expanding the SATA ports to 8, basically) it would be enough for formatting with ZFS, right?

For example, this "HighPoint RocketRAID 2720SGL" (150€) would be enough for my system?

Would it make any real difference if I attach the drives to a hardware-based RAID card, like the "Highpoint RocketRaid 4520" (450€), given I am going to use FreeNAS and ZFS as file system (probably RAIDZ).

Thanks a lot again!

 

[MooCow]

You should read into using that SSD for ZIL with ZFS... installing FreeNAS on the SSD alone might be a waste as the OS has a pretty small footprint.

I think you might be able to get away from needing to buy a high end PCI-e SAS controller with a write cache by utilizing ZIL.

However I'm must new to building my own FreeNAS box as well, so I may not have the answers to any of your questions. I'm sure someone can offer clarification.

I just bought a used IBM ServeRaid br10i card from the For Sale section. Its just an entry level card and far from anything high end. I got this cable to go with the SAS card : http://www.bhphotovideo.com/c/produ...nt_INT_MS_1M4S_Internal_Mini_SAS_to_SATA.html

It looks pretty damn nice as it is sleeved and its one of cheaper cables on the site. It says its compatible with Highpoint rocket raid controllers, but it works fine with my IBM ServeRaid (LSi) card.

On a somewhat unrelated note, an issue I am running into is that FreeNAS is not recommended to be run in a Virtualized environment. Virtualizing FreeNAS is a great way to learn ZFS, but I have been warned against using it as a dedicated/production NAS and even against personal home use. Reasons being - complicated overhead of ESX's own method to write to the disk, and no S.M.A.R.T. monitoring of drives unless if you use passthrough to assign another SAS controller to the FreeNAS guest OS. That's a short list to start off with, I'm leaving out a lot more reasons as I don't want to derail the thread.

 

[_Gea]

Please compare a more pro-like setup - especially with ZFS in mind:

- SuperMicro X10-SL7-F (about 250 Euro)
comes with 32 GB ECC RAM max, Ultrafast LSI HBA (=LSI 9207) controller, two additional fast pci-e slots,
2 x Intel Nics but they are very new, sometimes not yet supported (additional PCI-e Intel Nic may be needed, add 20 Euro)

- add cheapest Xeon and at least 8 GB RAM
- For a NAS, you do not need a ZIL or a Cache SSD, use cheapest boot-disk > 20 GB
even 16 GB USB sticks are possible with FreeNas or OmniOS (newest ZFS, Solaris based where ZFS development is done ) - best in a ZFS mirror

Avoid Highpoint, use LSI HBA all the way.
With such a config, you can either build a highseed NAS or even an All-In-One Config
(ESXi with virtualized SAN and other virtual machines like OSX, Linux, Windows or Solaris) where you need the extra capability of such boards.

 

[MooCow]

I too heard not to use anything from Highpoint.

Highpoint thread from late last year: http://forums.freenas.org/threads/highpoint-controller-info.8217/

Your best bet is to get a rebadged LSi card sold under IBM, or a Dell Perc 5i. Keep in mind the Dell Perc cards may not work well in some Intel motherboards and they need to be modified by taping off or painting over certain pci-e gold connects. I think this is too much trouble if you can afford to go with a rebadged LSi w/ write cache features.

This guy is selling SAS cards in the For Sale section and I just received my card from him yesterday - http://hardforum.com/showthread.php?t=1772915&highlight=
You most likely can't find his card for under $200 anywhere else.

 

[mini-me01]

Save yourself some trouble and buy a LSI 6Gb HBA from the start. The IBM BR10i is a 3Gb HBA and is not going to work with drives over 2TB.

I would reccomend buying an LSI 9210-8i/9211-8i or their oem branded equivalents from IBM (M1015) or Dell (H310,H2XX). I have a Dell H310 running LSI firmware and there is no need to tape over any PCIe contacts, this is the first I have heard of that

 

[sy5tem]

Hello,

i must first say that Proplus as a really good point regarding raid5 ... with large array,

then i must say that FREENAS is not that superb, its really hard to tweak, its really finicky about hardware, without zil its slow ass, require lots of ram 8GB + ,e.t.c....

with today price of ram and general hardware, you should simply use something like windows server, so you can use it as a storage AND hyper/vmware virtualisation host, so you can get more for your money...

the only thing here is getting a nice RAID 6 card, on a PCIE bus, and for sub 400$.. i only see 2 reliable choice atm,

3ware 9650SE, sata2, but as raid 6 and does 300MB to 450MB / Seconds on RAID6 and works with WD RED (no TLER problem)

ARECA sata2 1231ML ( Rev, B -max 2GB cache ) as large cache and raid 6, can get 450MB to 800MB / seconds

both card are servers card, with BBU unit e.t,c

with this setup you don't really need an SSD, and you got raid 6, for NAS / virtualization...

just my 10 cents


in my opinion you really don't need SATA 3 raid .... its way too costly and will not bring THAT much speed. compared to large cache areca raid card for example

 

[TeeJayHoward]

Main use: Homeserver for streaming music (flac), videos (mkv) and games (7z)
Budget: ~700-800€ (HDDs not included)
Hard disks: 8xHitachi 7K4000 4TB
RAID: 5 or 6 (not decided yet)
SO: FreeNAS 9.1 x64

Main specifications of the system:

CPU:Intel Core i3-3220T (~115&#8364
MB: Asrock H77M-I (~90&#8364
RAM: 2x4GB Samsung DDR3-1600 CL9 Low Voltage (~60&#8364
SSD: Crucial M4 64GB (~75&#8364
HDD: 8xHitachi 7K4000 4TB
PSU: Seasonic SS-300SFD (40&#8364
FANS: 2xGentle Typhoon 120mm@800rpm (~25&#8364
RAID: ? (~150€?)
CASE: SilverStone DS380 (not released yet) (~200€?)

1) The SilverStone DS380 is a μITX case. The Asrock H77M-I is a μATX motherboard. It won't fit.
2) If you're doing ZFS, you want ECC RAM. It's not necessary, but it is a VERY good thing to have if you care about data integrity. That means a Xeon processor (if you're sticking with Intel). That means a professional motherboard. I'm partial to SuperMicro, myself.
3) A 300W PSU for 8x3.5" drives is cutting it REALLY close. Try one of Seasonic's 400W ones.
4) 800RPM fans are insufficient to cool densely packed hard drives. I had to upgrade mine to 1200RPM fans. Remember that every manufacturer flat-out LIES about their CFM and dBA ratings.
5) As for controller cards, just get whatever is supported. SATA II vs. III is irrelevant unless your array is SSD. You're looking at a maximum speed of around 1.2GB/s. Realistically, I'd be surprised if you exceeded 1GB/s, and that's transferring files around inside the box. Expect 115MB/s transfers to/from the box, limited by the 1GbE NIC.
6) With ZFS, you want 1GB of RAM for every 1TB of storage. In a 8x4TB RAIDZ, you're looking for a motherboard that'll do 32GB of RAM.
7) If this is just a ZFS fileserver, the OS drive is irrelevant. There's no need for an SSD there. Heck, there's no need for a proper drive. Stick a thumbdrive in a USB slot and install to that.

Your requirements are very similar to what mine were when I built my NAS.
Case: $75, 8-bay. Fractal Design Arc Mini, maybe? Comes with some nice, quiet fans.
CPU: $215, something with ECC support. E3 would be my recommendation.
Mobo: $155, something with ECC support. SuperMicro mATX
RAM: $300, 4x8GB ECC UDIMM
PSU: $120, Fanless 400W Seasonic. If you don't care about noise too much, feel free to replace this with a cheaper 120mm fanned one.
=====================
Total: $865. It's μATX instead of μITX, but it's small, quiet, and powerful. All you need to do is pick up a nice used HBA off eBay, and you're set for about $1000USD (750&#8364

That's basically what I did, except I used the onboard 6xSATA ports for my drives instead of buying an HBA, since I only had 6 drives. Pic:

 

[Slackusr]

I am currently spec'ing out the same kind of setup. My rig will be similar to TJ's above. As of right now I am going to try the all-in-one ESXi/OpenIndiana VM route. So far I am looking at the following:

Mobo: Supermicro X10SL7-F (mainly for the built-in SAS controller, appears to be easily flashable to IT mode)
CPU: Intel Xeon E3-1230 V3
RAM: 32GB Kingston ECC
Drives: (8) 3TB Western Digital Red
Case: Cooler Master N400 (holds 8 HDD plus 3 SSDs) Although I am checking out the Fractal Design Arc Mini R2 as well. http://www.amazon.com/Cooler-Master-N400-Computer-Meshed/dp/B00DOZHJU2/ref=wl_it_dp_o_pC_nS_nC?ie=UTF8&colid=3VZOAYAN2VL9Z&coliid=IQ11QDSHE5AL1

I am still up in the air on SSD needs and choices.

 

[Flintstone]

Mobo: Supermicro X10SL7-F (mainly for the built-in SAS controller, appears to be easily flashable to IT mode)

The nice thing about using a cheap IBM 1015 adapter is that it's easy to move to another system if needed (say you need two adapters in a system for a short time). I have at least 7 of them...

BTW: I've bricked HBA adapters in the past so be thorough when flashing...

 

[MooCow]

I have a Dell H310 running LSI firmware and there is no need to tape over any PCIe contacts, this is the first I have heard of that

I'm referring to the old school Dell Perc 5i's. They did not work in all motherboards which would turn away most people except there was an easy work around hack to make it more compatible.

Using this method would obviously limit your intentions down to building a personal home NAS or for study in a practice lab.