New ZFS build - Small, silent and appliance like

I

ironicbadger

n00b
Joined
Nov 8, 2013
Messages
28
I've just suffered catastrophic data loss across my 12TB array which was running using SnapRAID and MHDDFS due to a faulty RAID controller and my inability not to tinker. I am therefore building a new system which is designed to be as 'set it and forget it' as possible...

OS: FreeNAS (latest)
Motherboard: ASRock C2750D4I Octa-Core Avoton Intel Atom with 8x sata ports
RAM: Kingston 16GB ECC
Case: LIAN LI PC-Q25B
Power Supply: Seasonic TFX-350 Fully Modular SFX
Fans: Replacing stock fans with Noctua, no CPU fan.

Drives - 6 or 7x 3TB mixed brands from Hitachi, Toshiba and WD Reds.

I'm going ZFS and am thinking I'll setup the drives in one big Z2 pool. This should give me with 7 drives ~10TB of usable space. I know 6 drives would be optimal but this box will only be connected via gigabit and therefore I don't think I'd notice any real-world benefit in performance by running the optimal z2 6 drive config. I would notice the missing 3tb more I suspect. I have a 128gb SSD lying around which I thought I might be able to use in some cacheing capacity, but I don't fully understand if I'd benefit from doing this or not.

Is ZFS even the best option for this box? It's going to be storing mostly media files (TV, Movies and Photos etc) as it's a home storage box. My super important stuff is backed up to a secondary Synology NAS with mirrored 3TB drives and then off to Crashplan and my parents house via BitTorrent Sync. I've used unRAID in the past and their new v6 is looking pretty decent with the Docker support but I can't decide...

ZFS or.........?
 
I'm just going comment generally based on my experiences with FreeBSD and ZFS...

Any reason why you'd pay a premium for Avoton? You're much better off getting a Celeron and/or (perferably) an i3 CPU both in terms of performance and costs. That said, there has been reports about instability using the Avoton platform so I'd be a bit hesitant going in that direction. You'd also want to avoid the Marvell controllers used on the ASRock board.

ASRock E3C224D2I + Intel Core i3-4130T --> 328$
SIIG SC-SA0L11-S1 (ASMedia AHCI controller card for the additional 2 ports) --> 24$

Are you sure that an SFX PSU fits in that case?
http://www.newegg.com/Product/Product.aspx?Item=N82E16817371058
Should do fine otherwise and at a very good price...

I would be very careful mixing drives, you'll get reduced performance by doing so. That said, I've had great experience with Toshiba's DT01ACA-series but that doesn't mean that they'll never break.

If I were you I'd probably go for FreeBSD 10.1 as it incorporates a lot of desired fixes and features which 9.3 lacks (that's what FreeNAS is using at the time of writing) however that's CLI only. Since you're only going to use (from what I can tell) NFS, Samba and possibly rsync it's not an issue really. If you are dead set on a WebUI you can either use the nightly builds (they'll be fine) for FreeNAS and/or NAS4Free.

Since you're "only" on 1Gbit and it'll probably serve you well I'd skip the SSD cache, it just adds complexity and more stuff that can brake.
//Danne
 
1st there is no set and forget solution.
you must care about the state of your hardware from time to time.
It would be good if the appliance can send email or push alerts to your smartphone.

If you look at very reliable solutions, you must check
- filesystem quality
- raid quality
- OS quality
- hardware quality


regarding filesystems
there is no doubt that new gen filesystems like btrfs, ReFS or ZFS
who offer CopyOnWrite and realtime/ end to end checksums are far better than
ext3/4, hfs, ntfs or xfs.

best of them regarding performance, features and reliability: ZFS


regarding raid
there are three options
realtime hardware-raid
realtime software-raid ex ZFS
non realtime /backup solutions like snapraid

for a pure mediaserver, a backup solution can be ok as inactive disks can sleep and you can add single disks.
But if you need realtime or performance or set and forget, prefer realtime solutions like ZFS

For set and forget, add a hotspare that can replace a faulted disk automatically.


regarding OS
in case of ZFS, you can use
Oracle Solaris or a free fork like OmniOS (this is where ZFS comes from with unique WWN disk namings, mirrored ZFS system pools and working hotspares, my preferred OS)
BSD (often not as fast as Solaris especially with NFS and SMB but widely used)
Linux (ok if Linux is a must)


Hardware quality
I would avoid solutions based on Avaton because they are quite expensive, not as fast as i3 or Xeons,
mostly with limited expandability. The included Sata ports based on Marvel chips are not as fast or well supported
like regular Sata AHCI ports or HBA solutions from LSI

I would prefer a solution based on Supermicro with IPMI remote management, i3..Xeon with ECC RAM and optional an LSI HBA like
http://www.supermicro.nl/products/motherboard/Xeon/C220/X10SL7-F.cfm (with LSI HBA) or
a similar board without the extra 8 port LSI host bus adapter see http://www.supermicro.nl/products/motherboard/Xeon3000/#1150
 
Last edited:
There's no point in going for an integrated LSI controller if he doesn't need 8+ drives, Intel AHCI is actually faster anyway however it doesn't support port multipliers etc.

Regarding brand I think ASRock is going to be fine, don't really see the greatness of Supermicro. If a regular tower case is fine I'd say buy a prebuilt by Fujitsu, Lenovo or possibly HP. Usually much cheaper in the end and at least Fujitsu and Lenovo servers are very silent.
This however might get a bit tricky if you need a lot of drives...
//Danne
 
8 drives is the target for this system including the SSD (7x data drives inc parity). with regards to the avoton vs the xeon the entire reason i'm thinking of the atom board is because it doesn't support vt-d (neither does an i3 granted) and there in lies it's appeal. i won't be tempted to tinker with it.

with regards to cost, that's another interesting point as most ITX boards I can find which are traditional socket based (xeon etc) only have 6 sata ports meaning i'd need to spend more on a HBA card. as with most things in life, there is no one right answer is there? ugh. wish there was.

as for buying a pre-built server I find a particular joy in building a system so that's not happening! :D

i'm stuck weighing up between ZFS and unRAID. Obviously there is a lot of performance difference between the two plus a lot of pro's and con's regarding the availability of data if the 'array' exceeds it's fault tolerance.
 
@ ironicbadger
So it's better to have a crap controller integrated? I've linked to well working controller that costs 24$, you can't get something much cheaper than that which gives you 8 ports to use. You don't really need a Xeon at all unless you plain to do transcoding.
You do realize that ZFS is a file system and unRAID is a distribution that uses reiserfs, xfs or btrfs?

As Gea_ said, there's no really "set and forget" but depending on what setup you choose it doesn't need much attention.
//Danne
 
What makes the integrated controller crappy?

Thanks, I do appreciate the differences between a distribution and a filesystem.
 
Poor performance/reliability and (driver) support
In short you want a non wonky AHCI controller (Intel and ASMedia are the only ones I know of) or LSI (LSI2***-series).
//Danne
 
the supermicro board suggested by _gea seems to be a uATX board rather than my requirement of mini-itx. does such a board exist at this size?
 
I'll be frank about FreeNAS. I've lost an 8TB raidz1 array twice now during server moves. The first time I was able to recover the array using some command line magic from the internets. The second time I lost all of it, this was a week ago today. Both times I exported the array before a full shutdown. The new server fails to import. Moving back to the old server fails to import. Eventually received a "corrupted" data message.

Since this is all departmental "nice to have" backup data it's not a big deal. I would never in a million years trust FreeNAS with meaningful data. Never. Too flaky and easy to break. With that said, I rebuilt the array on FreeNAS again because it just works and doesn't cost $700 in licensing.
 
_Gea didn't mention it directly as it's his baby but Napp-it "just works" too.

Personally I use it with OpenIndiana and not OmniOS as I need a GUI and run some windows software through wine on the server.
 
@ oscadd
This is one of the reasons I recommend using FreeBSD 10.1 instead of 9.X....

@ ironicbadger
It's the same mobo I mentioned in the second post...

//Danne
 
diizzy said:
@ oscadd
This is one of the reasons I recommend using FreeBSD 10.1 instead of 9.X....

//Danne
Click to expand...
Other than being the new thing, what are the clear benefits of 10.1 versus 9.X? I've been considering a FreeNAS build, and only recently have I started to see more discussion of issues with FreeNAS.

Edit: Also what are the reports of instability on Avoton?
 
Last edited:
ZFS has been much improved in general, base compiler is LLVM+clang instead of an acient version of GCC. Overall performance is much better and subsystems are also improved quite a bit.
https://www.freebsd.org/releases/10.0R/relnotes.html
https://www.freebsd.org/releases/10.1R/relnotes.html

Avoton seems to have buggy bioses causing crashes and power management seems to be so-so. Haven't ooked into it too much as I find the platform quite uninteresting, i3s are a better choice and if you want a low-power one go for the T-version(s) I mentioned earlier.

//Danne
 
I really, REALLY like having IPMI on my server equipment. Don't have to fiddle with a keyboard/monitor/mouse and you can do everything over the network.

For your core OS, I like OmniOS if you want to do a ZFS dedicated setup. FreeNAS I like if you want ZFS and also a NAS appliance where you can run plugins and lots of other features. For example, I have OmniOS & napp-it for hosting my ESXi datastores, and FreeNAS running my Plex/SABnzbd/SickRage/OwnCloud/CouchPotato jails.
 
You must log in or register to reply here.
Back
Top