• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ZFS build - Optimizing config for workload

I

icanhazapple

n00b
Joined
Mar 19, 2012
Messages
15
Well, 2015 is the year. I'm finally going to pull the trigger on a FreeNAS build. 5x4TB WD Red in RAIDZ2, 32GB ECC, Xeon E3-1220. 1GBe between server and single client. Based on the insight I find within this thread, I should be up and running within a week.

Here's where I'm looking for some additional insight, based on my intended workload. Client will be a single Windows 8.1 machine - iSCSI will be used to access the datastore. Adobe Lightroom/Dropbox/iTunes/Backblaze/Torrent are the primary applications. Right now the dataset is 7TB in size.

Why iSCSI? - Only way Backblaze will touch the datastore, I think.
ZIL - Leaning towards not necessary.
L2Arc - Since we're only using a 1GBe link, not necessary.

NTFS over iSCSI on ZFS Datastore - the whole point is to protect my data. Are there any additional gotchas with using NTFS on ZFS?

Am I missing anything here? Basically 10GBe might be relevant in 2020 - we can revisit the ZIL and L2Arc questions later (Guy at my local datacenter has a pretty big ZFS datastore, so I can encrypt and copy my stuff while I re-architect the zpool). But based on what I've read, this should be adequate for now (a bit overkill, but I want this to last for 10 years)

Thanks!
 
While I can't be super helpful with the specifics of your question, as a new user of FreeNAS 9.3, I can at least give you my basic experience having setup a box last week.

FX8320
24GB ECC
6x4TB Seagate NAS in RaidZ2
1x GbE
Fractal Design Define R4
Panamax MB850 UPS

Mostly a torrent and plex box, although soon will be adding 2x drives in a mirror to handle user data for my wife and I. I use the plugin for transmission and just connect all the clients in the house using the remote GUI clients. Don't see any particular reason why you'd want to load up the network interface with torrents when you don't have to.

I was able to maintain a 110MB/s sequential write onto the server over network while also copying data internally at ~90MB/s (limit of the internal drive doing the copying, which was a 2TB WDC green drive). This was just copying to a standard Windows share.

It was a lot of fun to put together and both easier and harder than I thought it would be. My only *nix experience is 16 years ago in high school doing tech support for a dialup ISP. Setting up the UPS was easy, just chose a generic UPS driver as the one I have is obscure and wasn't listed. Only issue I really had was getting the NTFS drives mounted to copy data onto my zpool. Documentation isn't exactly clear (fuse is not loaded by default) but was corrected by finding the one command to run on cli.

And the PSU failed while I was copying data the first time, loud pop and ozone smell. Luckily didn't damage any of the hardware and had a spare to get back up. Not really FreeNAS's fault :)

edit: I should also mention that there was a vendor on ebay listing a bunch of Intel X-25e 32GB drives brand new for $50 obo the other day. I alllllmost bought some for a ZIL even though I knew I didn't need it. My use case is almost entirely streaming movies and TV so an SSD cache is just an extravagance that I may set up at some point just to learn how.
 
General rule of thumb of ZFS is "power of two disks, plus parity." For five disks, you should use Z1 (4+1) or Z3 (2+3)..and shouldn't use Z1 IMHO. If you really want Z2, try 2+2 or 4+2. This is best practice due to the way ZFS writes across a vdev. It will work as Z2 (3+2), but may not provide optimal performance.


For a quick reliability calc, try this: http://www.servethehome.com/raid-calculator/raid-reliability-calculator-simple-mttdl-model/ I'm a pessimist so I use MTBF = 36.5K and VRS = 10MB/s. The Reds you're looking at are NER = 10^14, Capacity = 4TB, and Sector Size = 512B.


If your workloads are more bandwidth focused than IOPS, a single vdev should be fine with one client and only GbE. Here are some recent ZFS benchmarks: https://calomel.org/zfs_raid_speed_capacity.html However, if you need more IOPS, you'll want to stripe several vdevs in your pool (i.e. like RAID-60, but multiple groups of Z2/3). You could also go with a RAID10 array if you need even more IOPS. The benchmarks above are bandwidth focused


The ZIL always exists, it's merely how its handled. It's usually only talked about when it's offloaded to a dedicated SLOG; typically an SSD. It's still there even if you don't offload it, it just puts more strain on the pool. If you have a lot of sync writes, you'll want one. You can pick up a Intel DC3500 80GB for less than $100.


Do some research and only use known good SATA ports if you want any performance from the pool. Towards the end of the page, the benchmark address this.
 
gunderwood and Abhaxus, thanks for your replies!

gunderwood - thanks for your calculations, that is helpful - I usually am very conservative with MBTF calculations, so the baseline numbers are appreciated.

Abhaxus - thanks for sharing your experiences, that PSU frying would have given me a heart attack.

Hmm, I'll probably pick up a S3500 for the ZIL then, that's pretty reasonable.
 
How far will your NAS be away from your workstation? If it's close, there isn't any reason NOT to go with 10GbE, as cheap as cards are on fleabay.
 
Any Intel offer (X520/540 or with same chips from Dell, HP, IBM,SuperMicro, ..) is perfect.
Any other may give you headache at least with Solaris/OmniOS
 
_Gea said:
Any Intel offer (X520/540 or with same chips from Dell, HP, IBM,SuperMicro, ..) is perfect.
Any other may give you headache at least with Solaris/OmniOS
Click to expand...

I've picked a bunch of Brocade 1020s on the cheap. There aren't any BSD drivers, but I've read a lot about them being used under OmniOS w/o issues. I use them in my home lab for iSCSI connection between ESXi hosts and my storage. A pair of those and an ACTIVE Twinax DirectAttach cable should set you back less than a $100. Haven't looked into many of the others, but I've seen a ton of the older Mellanox 10GbE cards drop in price lately as well.
 
They work due the efford of Sun with Opensolaris.

Now we have a free and open Solaris fork with Illumos with less resources.
Do not expect too much bugfix support from Illumos with non mainstream hardware (=mainly Intel)
 
You must log in or register to reply here.
Back
Top