ZFS & Encryption - Information overload

Statement :
I like to keep my data encrypted; all my laptops and workstations are, so i think my fileservers should be as well
First fileserver was on MS Vista (don't ask) with all drives truecrypted, right now i'm using a NL40 microserver with nas4free GELIed individual drives.

I will be upgrading next week to a 12drive, dual RAIDZ2 Xeon setup based on ESXi.
Before committing to an OS, i thought of collecting my different options, and hope that this will help put a little clarity into all of this.(strict homeserver build)

General requirements :

• needs to run under ESXi (vmxnet3 for 10GBe)
• needs to support ZFS
• needs encryption support
• file-sharing via CIFS
• easy administration would be a boon

Solaris Express 11
+ Encryption support integrated into ZFS
+ Kernel CIFS
+ napp-it can added for easy administration
- No updates support without paid Oracle license
- closed-source ZFS

My takeaway :
Ah, this would have been easy... Although native encryption support isn't deemed to be the fastest, it is all integrated and would make for the cleanest solution.
Unfortunately, having an OS installed with no possibility for non-paid updates, and incompatibility with opensource ZFS (i would have to use ZFS28+ for encryption support) ... makes this a no-go

OmniOS/napp-it
+ Well-tested under ESXi
+ A-grade ZFS support
+ Kernel CIFS
+ Easy administration via napp-it
- Encryption needs to go through ZFS twice via spare-encrypted ZPools

My takeaway :
I love Gea's work on napp-it. This would be my first choice, as everything falls in place, except for fulldisk encryption. If i understood correctly, Solaris-based OS cannot do full-disk encryption, so :

• one has to make a first zpool
• fill that with files
• encrypt those files via lofiadm as block devices
• make a new zpool from those block devices

This does seem quite ... complicated for full-pool encryption. Also, to use the full HD capacity, the whole first zpool would need to be filled to the brim with encrypted files ... i wonder if there will be a performance impact.
If it weren't for encryption, i wouldn't have opened this thread, but gone for OmniOS straight away.

FreeBSD 10
+ Virtualization enhancements compared to 8 and 9
+ vmware tools available
+ GELI-based encryption
- Samba (no kernel CIFS)
- no easy administration interface

My takeaway :
Encryption-wise this seems far simpler than OmniOS : i need to GELI the drives first, and put a zpool on top of it. Administration-wise I'm not scared of the command line, but there's no integrated niceties like disk-failure email reporting, SMART status ... or other boons that come with nice admin panels. Still, BSD 10 feels like a strong contender.

nas4free
+ GELI-based encryption
+ easy administration interface
- none of the BSD10 enhancement on VM support (still based on BSD9, iirc)
- devs are not adamant about VM support
- Samba (no kernel CIFS)

My takeaway :
I use nas4free on my HP Microserver, and really happy with it:

• easy GELI encryption via GUI
• the whole configuration can be backed up to a text file (love that feature!)
• integrated ZFS tuning

Now the big question is how viable nas4free is in a VM ... this asks for testing.

tl;dr
Seems i cannot have it all :

• Great ESXi, GUI support, but lacking easy encryption : OmniOS
• Great ESXi, encryption support, but lacking GUI : FreeBSD10
• Great GUI, encryption support, but unsure of ESXi : nas4free

As things stand now, i will first test nas4free in ESXi and stick with it if it works fine.

Anything i got wrong with the above? Suggestions? I'm still on the fence

seku said:

Statement :

FreeBSD 10
---
My takeaway :
... Administration-wise I'm not scared of the command line, but there's no integrated niceties like disk-failure email reporting, SMART status ... or other boons that come with nice admin panels. Still, BSD 10 feels like a strong contender.

Click to expand...

All the integrated niceties are available on the CLI. man smartctl. Most GUI interface merely read data from a CLI application.

ezno_matrix said:

All the integrated niceties are available on the CLI. man smartctl. Most GUI interface merely read data from a CLI application.

Click to expand...

Yes, I would have to write the scripts to check ZFS and disk status myself. No impossible task, it's just that they all come prewritten and tested with the GUIs. i'd feel like doing double work.

But you're right, i could give that a try.

No scripting is required for SMART. SMART runs as a daemon (smartd) and is controlled by smart.conf file. All your settings go there. Here is an example of a smart option:
# Monitor all attributes except normalized Temperature (usually 194),
# but track Temperature changes >= 4 Celsius, report Temperatures
# >= 45 Celsius and changes in Raw value of Reallocated_Sector_Ct (5).
# Send mail on SMART failures or when Temperature is >= 55 Celsius.
#/dev/sdc -a -I 194 -W 4,45,55 -R 5 -m [email protected]

I don't know much about zfs to comment on what monitoring is require. I use mdadm, which automatically sends email if something needs my attention. No need for scripting, these are all built into the daemon. I would consider ZFS very limited if it did not have basic alerting built in.

was reading up a bit about the ZFS part.... seems like ZFSd (the daemon) is not shipping yet with FreeBSD10. Information is sparse, but supposedly they want to ship it with 10.1... so i'd have to do with cronjobs or the like...
not the most elegant solution, but well

EDIT : maybe this could do as well : http://zfswatcher.damicon.fi/