Help me build a ZFS file server

E

EnderW

[H]F Junkie
Joined
Sep 25, 2003
Messages
11,249
I'm considering building a new ZFS file server to replace my aging WHS.
I already posted in the OpenSolaris thread, but thought I would make a new thread.

This would be used for home use - backup and media storage/streaming to about 5 other computers.
I would like this to last me around 5 years.
Planning on FreeNAS as the OS right now.


Here is what I am thinking
Chenbro RM41824M2
SUPERMICRO MBD-X9SCM-F-O
Intel Xeon E3-1230 Sandy Bridge
32GB of Kingston 8GB ECC DDR31333MHz KVR1333D3E9S/8G
3 x IBM ServeRAID M1015
Intel 320 80GB SSD for boot drive

what hard drives?

would like to start with 8 (2TB) drives in RAID Z2, add a second set of 8 (3TB) later on, and a 3rd and final set of 8 (4TB) after that
probably around 72TB of drives in the end

I read that ZFS needs about 1GB of RAM per TB of storage, so I may need to get a motherboard with more memory slots?
SUPERMICRO X9SRA
X9SRi-F
or maybe add a SSD for caching?
Intel 710 100GB SSD
 
Last edited:
As an eBay Associate, HardForum may earn from qualifying purchases.
You dont't need a Intel 320 as boot drive, go for mirrored usb sticks instead.

ZFS doesn't require 1GB memory per TB, you will be doing fine with just 2GB. More is better, but mostly for reading and if you have a lot of concurrent IO.
 
1GB per TB is for the deduplication tables.

The more ram you can throw at zfs, it will use for caching disk reads.

Toss up between whether you expect to benefit more from more cache, or dedupe.
 
I wouldnt bother with dedupe if this is for backup/media storage. 32 gigs should be plenty, buy as much as you can at a good price and beyond that you should be fine.
 
Definitely more ram than you need since this is just for mass media storage. I'd go with a maybe just a single stick 8GB, that way you can always add more later up to 32 if you like. As said...the boot drive is also an unnecessary expense. A cheap laptop 2.5" drive, or two for a mirrored boot, is plenty.

Lastly the E3-1220 is cheaper than the 1230 so might as well save a few bucks there too. ZFS won't make good use the hyper threading anyway so little point in going to the 1230.
 
I just did a similar build, went with Hitachi 5k3000 2TB disks. I have been very happy with the solution so far.

See this post for more info
 
EnderW said:
I'm considering building a new ZFS file server to replace my aging WHS.
I already posted in the OpenSolaris thread, but thought I would make a new thread.

This would be used for home use - backup and media storage/streaming to about 5 other computers.
I would like this to last me around 5 years.
Planning on FreeNAS as the OS right now.


Here is what I am thinking
Chenbro RM41824M2
SUPERMICRO MBD-X9SCM-F-O
Intel Xeon E3-1230 Sandy Bridge
32GB of Kingston 8GB ECC DDR31333MHz KVR1333D3E9S/8G
3 x IBM ServeRAID M1015
Intel 320 80GB SSD for boot drive

what hard drives?

would like to start with 8 (2TB) drives in RAID Z2, add a second set of 8 (3TB) later on, and a 3rd and final set of 8 (4TB) after that
probably around 72TB of drives in the end

I read that ZFS needs about 1GB of RAM per TB of storage, so I may need to get a motherboard with more memory slots?
SUPERMICRO X9SRA
X9SRi-F
or maybe add a SSD for caching?
Intel 710 100GB SSD
Click to expand...

I am building a similar setup, so far I haven't been able to get the supermicro motherboard to recognize the m1015s.... :( I need to flash them on a different PC and see how that goes though.

Also, you may be better off with 10x2TB drives in a raidz2 because of the way ZFS stripes (so I am told) you want to have a total number of data drives that distributes 128 evenly.
for example 8x2TB has 6 data drives which means 128 / 6 = 21.333 (an uneven distribution)
if you use 10x2TB you have 8 data drives which means 128 / 8 = 16 (an even distribution)
or even a 6 x 2TB is 4 data drives which means 128 / 4 = 32 (even distribution)

I am no expert on this, but it should make a substantial difference in performance.
 
As an eBay Associate, HardForum may earn from qualifying purchases.
cbutters said:
I am building a similar setup, so far I haven't been able to get the supermicro motherboard to recognize the m1015s.... :( I need to flash them on a different PC and see how that goes though.

Also, you may be better off with 10x2TB drives in a raidz2 because of the way ZFS stripes (so I am told) you want to have a total number of data drives that distributes 128 evenly.
for example 8x2TB has 6 data drives which means 128 / 6 = 21.333 (an uneven distribution)
if you use 10x2TB you have 8 data drives which means 128 / 8 = 16 (an even distribution)
or even a 6 x 2TB is 4 data drives which means 128 / 4 = 32 (even distribution)

I am no expert on this, but it should make a substantial difference in performance.
Click to expand...
hmmm, I will have to re-plan this then
I wanted to add drives in stages as to keep initial costs down and have maximum storage at the end
 
Silenus said:
Definitely more ram than you need since this is just for mass media storage. I'd go with a maybe just a single stick 8GB, that way you can always add more later up to 32 if you like. As said...the boot drive is also an unnecessary expense. A cheap laptop 2.5" drive, or two for a mirrored boot, is plenty.

Lastly the E3-1220 is cheaper than the 1230 so might as well save a few bucks there too. ZFS won't make good use the hyper threading anyway so little point in going to the 1230.
Click to expand...
thanks everyone for the info on RAM
I'll probably do either 1x or 2x of the 8GB sticks so I have room down the road if I need it

I will probably keep the SSD as I already have a spare and I'm not sure what the mounting options are in that case so I may just have it laying in the case or velcro to the side
plus I want to keep my options open if I decided to go with another OS like WHS v2 or something

E3-1220 it is :)
 
what do you guys think about for a PSU?
this is going to be right by my desk so no jet turbine server room redundant units
maybe a Seasonic X series?
 
I'm probably gonna order some of this stuff up next week
at least the MB, CPU, and RAM

any last input?
 
Yes. Order me some parts too will ya? :)

You could always create multiple vdevs of say 5 drives (4 data and 1 parity). Then down the road, you could combine the vdevs. Just saying since I'm looking to do a ZFS build some time in the future, but the hard drive prices are scaring me away.
 
I hate to blatantly plug my product but you can download nexenta community edition for free from nexentastor.org
 
cbutters said:
IAlso, you may be better off with 10x2TB drives in a raidz2 because of the way ZFS stripes (so I am told) you want to have a total number of data drives that distributes 128 evenly.

I am no expert on this, but it should make a substantial difference in performance.
Click to expand...

Testing has shown it doesn't really matter for the normal case. It only matters if you trying to eek out every ounce of performance you can. Wish I could find the link again. Theory is cool....reality is what matters. If all you are doing is serving files witha gigabit connection..it should not matter at all.
 
Last edited:
Trepidati0n said:
Testing has shown it doesn't really matter for the normal case. It only matters if you trying to eek out every ounce of performance you can. Wish I could find the link again. Theory is cool....reality is what matters. If all you are doing is serving files witha gigabit connection..it should not matter at all.
Click to expand...
please post link if you find it
 
EnderW said:
motherboard, CPU, and RAM on the way

now for a case...
Click to expand...

How's the build going? I'm interested in doing something similar and am curious as to how things worked out for you. Can you post details / photos of your build?
 
You must log in or register to reply here.
Back
Top