Some NAS/ZFS questions

[fackamato]

Hi lads, long time lurker here!

So I want to set up a new NAS. I saw this post on ZFS benchmarks which leads me to believe that AMD Fusion is "good enough" for simple NAS usage for movie streaming etc when using ZFS.

First a question regarding ZFS, RAIDZ is like RAID5 with ZFS goodness, RAIDZ2 is like RAID6 with ZFS goodness?

I've put together a list of possible NAS solutions below, any feedback greatly appreciated! Main target is cost, then small size.

Cost per usable gigabyte is assumed that 4 HDDs are used in a sort of RAID5, so 6TB "usable" for the OS. "Powerful" means anything faster than Intel Atom or AMD Fusion.

Here are the alternatives I've found so far:

#1 ReadyNAS Slow, never see more than ~25-30MB/s over the network (single user)? From www.amazon.co.uk

Total: £441
Disk cost of total: 52%
Price per redundant TB: £60

* ReadyNAS: review here Netgear Ready Nas NV+ 4 Bay Gigabit Desktop Network Storage - Netgear £209
* HDDs: 4x Western Digital Caviar 2TB SATAII 64MB Cache 3.5-inch Green Internal Hard Drive OEM £58 ea, £232


#2 powerful AMD From overclockers.co.uk

Total 8GB RAM: £507 (incl shipping)
Disk cost of total: 47%
Price per redundant TB: £84

* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* Case: Lian Li PC-Q08B USB3.0 Mini-ITX Case - Black £86
* PSU: OcUK Swift 550W Silent Power Supply [PSU-457] £18
* MB/GPU: MSI 880GM-E41 AMD 880G (Socket AM3) DDR3 £50
* CPU: AMD Athlon II X2 Dual Core 250 3.00GHz (Socket AM3) - Retail [ADX250OCGQBOX] £42
* RAM: Kingston HyperX Blu 8GB (2x4GB) DDR3 PC3-10666C9 1333MHz Dual Channel Kit (KHX1333C9D3B1K2/8G) £49

#3 powerful Intel From overclockers.co.uk

Total 8GB RAM: £535 (incl shipping)
Disk cost of total: 45%
Price per redundant TB: £89

* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* Case: Lian Li PC-Q08B USB3.0 Mini-ITX Case - Black £86
* PSU: OcUK Swift 550W Silent Power Supply [PSU-457] £18
* MB: MSI 880GM-E41 AMD 880G (Socket AM3) DDR3 £50
* CPU/GPU: Intel Pentium G840 2.80GHz (Sandybridge) Socket LGA1155 Processor - Retail £60
* RAM: Kingston HyperX Blu 8GB (2x4GB) DDR3 PC3-10666C9 1333MHz Dual Channel Kit (KHX1333C9D3B1K2/8G) £49

#4 AMD Fusion Too slow for ZFS? Benchmarks here! From Amazon.co.uk .

Total 8GB RAM: £399 (excl shipping)
Disk cost of total: 60%
Price per redundant TB: £67

* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* Case/PSU: CIT CSCIT2003BLK Midi Case Prescott 24pin SATA 450W - Black £25
* MB/CPU/GPU: Asus E35M1-M Zacate Motherboard £98
* RAM: Kingston ValueRAM - Memory - 8 GB : 2 x 4 GB - DIMM 240-pin - DDR3 - 1333 MHz / PC3-10600 - CL9 - 1.5 V - unbuffered - non-ECC £36

#5 D-Link 4 bay NAS From Amazon.co.uk .

Total: £552 (ecxl shipping)
Disk cost of total: 43%
Price per redundant TB: £92

* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* NAS: D-Link 4-Bay Network Storage Enclosure - D-Link £312

#6 D-Link 4 bay NAS From eBay

Total: 515£ (ecxl shipping)
Disk cost of total: 45%
Price per redundant TB: £85

* HDDs: 4x Hitachi Deskstar 5K3000 2TB 3.5" SATA3 6Gbs Hard Drive £232, £58 ea
* NAS: D-LINK DNS-343, 4-Bay Network Storage Enclosure with OL £283

#7 HP ML110 G6 from Amazon.co.uk. Max 4 HDDs, you need to buy a SAS cable, HDD bay & controller to fit more. Also 3GB RAM only.

Total: 532£ (ecxl shipping)
Disk cost of total: 45%
Price per redundant TB: £89

* Server: HP ProLiant ML110 G6 - Server - micro tower - 4U - 1 x P G6950 / 2.8 GHz - RAM 1 GB - HDD 1 x 250 GB - Gigabit Ethernet - Monitor : none £225
* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* RAM: HP FX699ET 2GB (1x2GB) DDR3-1333 ECC RAM (Z400 / Z600) £67

#8 HP ML110 G6 From ebay.ie . 4GB ECC RAM, 1 x Intel Pentium G6950 / 2.8 GHz ( Dual-Core )

Total: 478£ (ecxl shipping)
Disk cost of total: 48%
Price per redundant TB: £80

* Server: ProLiant ML110 G6 Intel G6950 Dual Core 2.8GHz / 1GB RAM / 250GB Non Hot Plug SATA / DVD-ROM / 300W PS /
Manufacturer Part No: 578930-035 £196
* HDDs: 4x Hitachi Deskstar 5K3000 2TB 3.5" SATA3 6Gbs Hard Drive £232, £58 ea
* RAM: 4GB, 2x HP FX699ET 2GB DDR3 1333 MHz RAM Module 500670-B21 NEW £50, $25 ea

#9 Fractal Design R3, Intel Pentium 2.8 From Overclockers. Powerful! Expandable (8 HDD bays), very good case. 6 SATA, space for HBA or controllers

Total: 509£ (ecxl shipping)
Disk cost of total: 47%
Price per redundant TB: £84

* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* Case: £75
* PSU: [url=http://www.overclockers.co.uk/showproduct.php?prodid=CA-000-LA]Lepa N-Series 400W Power Supply £25
* MB: MSI H67MA-E35 Intel H67 (Socket 1155) DDR3 Micro-ATX Motherboard - (Sandybridge) **B3 REVISION** £60
* CPU/GPU: Intel Pentium G840 2.80GHz (Sandybridge) Socket LGA1155 Processor - Retail £60
* RAM: Kingston HyperX Blu 8GB (2x4GB) DDR3 PC3-10666C9 1333MHz Dual Channel Kit (KHX1333C9D3B1K2/8G) £49

#10 Fractal Design R3, AMD From Overclockers. Powerful! Expandable (8 HDD bays), very good case. 6 SATA, space for HBA or controllers, RTL8111E NIC

Total: 486£ (ecxl shipping)
Disk cost of total: 49%
Price per redundant TB: £81

* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* Case: £75
* PSU: [url=http://www.overclockers.co.uk/showproduct.php?prodid=CA-000-LA]Lepa N-Series 400W Power Supply £25
* MB/GPU: Asus M4A88T-M AMD 880G (Socket AM3) PCI-Express DDR3 Motherboard £60
* CPU: AMD Athlon II X2 Dual Core 250 3.00GHz (Socket AM3) - Retail £60
* RAM: Kingston HyperX Blu 8GB (2x4GB) DDR3 PC3-10666C9 1333MHz Dual Channel Kit (KHX1333C9D3B1K2/8G) £49

 

[Yonzie]

You are correct about the RAID levels.

I'd go with one of the 'powerful' ones, but replace the case with a normal ATX case. The Antec 300 is a great case, if not the cheapest. £50 from your chosen store, saving you £36 while accepting 6 hard drives and having 3 ODD slots.

 

[s0rce]

I use the Fusion (e-350) for my little ZFS fox with 4x 2TB 5k3000 drives. I run a few services (sabnzbd, sick-beard, asterisk, apache). I routinely get 70-90 MB/sec (single client, I don't put much load on it, mostly just me and my girlfriend using it) when transferring and it works great for streaming movies etc. Uses about 50W idle. Basically exactly your system #4 except I'm using the ASRock board and I've got a IBM Br10i SAS HBA for the hard drives and a couple mirrored 16gig SSD for the root pool.

The AMD system might be able to run ECC memory if you choose the right mobo so that is something to consider. Also, consider your desire to tinker and play with virtualization, its cheaper to get a xeon system now then upgrade later.

edit: also I agree with Yonzie's point about the case, I originally went with a little mini-itx case but I couldn't adequately cool the 4x drives and it was a pain, unless you need to make it small (i dont, its in the closet anyways). I'm using a cheape Cooler Master Elite 343 Mini Tower mATX Computer Case with the mini-ITX mobo inside.

Also, don't get a cheap crappy power supply and have it give sub-standard power and possibly lead to HDD problems.

 

[s0rce]

also, one thing to consider is the network chipset the motherboard has, realtek can have flaky support on solaris if you plan to go that route. I've found the realtek 8111E to be pretty good but people have had issues with others but if you have a spare PCIe slot you can always grab an intel NIC.

 

[raiderj]

I think even an Atom board will give you "good enough" speed for movie streaming with ZFS. I wasn't that brave, went with a 6x2TB i3 ZFS build. See my build log here.

 

[fackamato]

Thanks a lot guys. I've updated the first post (well, in 5 minutes) with a few more powerful choices, with a larger case (up to 8 HDDs without using the 5.25" bays).

More ZFS questions:

* Can you migrate from RAIDZ1 to RAIDZ2 online, or at all?
* Does ZFS benefit from 8GB of RAM if you don't use any deduplication or any other advanced features? (just RAIDZ* serving files)

Thanks!

edit: Added these:

#7 HP ML110 G6 from Amazon.co.uk. Max 4 HDDs, you need to buy a SAS cable, HDD bay & controller to fit more. Also 3GB RAM only.

Total: 532£ (ecxl shipping)
Disk cost of total: 45%
Price per redundant TB: £89

* Server: HP ProLiant ML110 G6 - Server - micro tower - 4U - 1 x P G6950 / 2.8 GHz - RAM 1 GB - HDD 1 x 250 GB - Gigabit Ethernet - Monitor : none £225
* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* RAM: HP FX699ET 2GB (1x2GB) DDR3-1333 ECC RAM (Z400 / Z600) £67

#8 HP ML110 G6 From ebay.ie . 4GB ECC RAM, 1 x Intel Pentium G6950 / 2.8 GHz ( Dual-Core )

Total: 478£ (ecxl shipping)
Disk cost of total: 48%
Price per redundant TB: £80

* Server: ProLiant ML110 G6 Intel G6950 Dual Core 2.8GHz / 1GB RAM / 250GB Non Hot Plug SATA / DVD-ROM / 300W PS /
Manufacturer Part No: 578930-035 £196
* HDDs: 4x Hitachi Deskstar 5K3000 2TB 3.5" SATA3 6Gbs Hard Drive £232, £58 ea
* RAM: 4GB, 2x HP FX699ET 2GB DDR3 1333 MHz RAM Module 500670-B21 NEW £50, $25 ea

#9 Fractal Design R3, Intel Pentium 2.8 From Overclockers. Powerful! Expandable (8 HDD bays), very good case. 6 SATA, space for HBA or controllers

Total: 509£ (ecxl shipping)
Disk cost of total: 47%
Price per redundant TB: £84

* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* Case: Fractal Design Define R3 Midi Tower Case - Black Pearl £75
* PSU: Lepa N-Series 400W Power Supply £25
* MB: MSI H67MA-E35 Intel H67 (Socket 1155) DDR3 Micro-ATX Motherboard - (Sandybridge) **B3 REVISION** £60
* CPU/GPU: Intel Pentium G840 2.80GHz (Sandybridge) Socket LGA1155 Processor - Retail £60
* RAM: Kingston HyperX Blu 8GB (2x4GB) DDR3 PC3-10666C9 1333MHz Dual Channel Kit (KHX1333C9D3B1K2/8G) £49

#10 Fractal Design R3, AMD From Overclockers. Powerful! Expandable (8 HDD bays), very good case. 6 SATA, space for HBA or controllers, RTL8111E NIC

Total: 486£ (ecxl shipping)
Disk cost of total: 49%
Price per redundant TB: £81

* HDDs: 4x Hitachi Deskstar 5K3000 2TB SATA 6Gb/s 32MB Cache 5900RPM - OEM £240, £60 ea
* Case: Fractal Design Define R3 Midi Tower Case - Black Pearl £75
* PSU: Lepa N-Series 400W Power Supply £25
* MB/GPU: Asus M4A88T-M AMD 880G (Socket AM3) PCI-Express DDR3 Motherboard £60
* CPU: AMD Athlon II X2 Dual Core 250 3.00GHz (Socket AM3) - Retail £60
* RAM: Kingston HyperX Blu 8GB (2x4GB) DDR3 PC3-10666C9 1333MHz Dual Channel Kit (KHX1333C9D3B1K2/8G) £49

 

[TheLastBoyscout]

I use the Fusion (e-350) for my little ZFS fox with 4x 2TB 5k3000 drives. I run a few services (sabnzbd, sick-beard, asterisk, apache). I routinely get 70-90 MB/sec (single client, I don't put much load on it, mostly just me and my girlfriend using it) when transferring and it works great for streaming movies etc.

I am in the process of setting up an almost identical system (ASUS E35M1-I and 4x 2TB 5k3000) and I get about 85 MB/s when copying a 7 GB .iso file from a Windows host (direct connection, no switch).
I also ran the Intel NAS Performance Exerciser (all test enabled) and got 40.7 MB/s as the overall average throughput (first time I ran it, so I don't have a good comparison). There is another thread here about the NASPT.

More ZFS questions:

* Can you migrate from RAIDZ1 to RAIDZ2 online, or at all?
* Does ZFS benefit from 8GB of RAM if you don't use any deduplication or any other advanced features? (just RAIDZ* serving files)

No, a live migration from RAIDZ1 to RAIDZ2 is not possible, you have to copy...
ZFS does benefit from extra RAM as it automatically uses it as cache. It depends on your usage pattern if you actually benefit from a cache. If you're mainly transferring large files (video), the cache is not helpful. If you're repeatedly reading a set of many small files, the cache is really noticeable.

-TLB

 

[ChrisBenn]

HP Microserver would be a viable options as well, and should be priced pretty competatively