Cheap, scalable ZFS-base storage

W

w00dix

n00b
Joined
Feb 20, 2010
Messages
8
Looking for storage solution to run VMware workloads over NFS and standard SMB filesharing for network local hosts. I've been toying around with the idea of using Nextenta or OpenSolaris so as to get the most feature complete version of ZFS but Opensolaris device support isn't anywhere near as broad as linux. So if I want more than 6 drives on an ICH10R, I have to go buy an expensive HBA and dumb it down to JBOD mode.

That's a problem for me. By the time I use up 2 drives for mirrored boot and 1 for caching SSD I'll only have 3 left for storage. A 3 drive RAIDZ is certainly better than the performance of 1 drive but it seems I'm throwing alot of hardware at the problem for only marginal improvement.

ZFS gurus, help me out here. If I want to go beyond 6 drives on my onboard sata slots, are ebayed HBAs and expanders really the only option? Is it reasonable to start with this configuration and then come back with a hba to external storage in a tower or rack mounted chassis later and add that to my ZFS-based system?

Any recommendations or advice would be appreciated.
 
I bought a Dell SAS6/i card for $35 shipped on eBay. 8x SAS host bus adapter with 1068E chipset. Runs on OpenSolaris just fine. 2x drives in rpool (mirror) and 6x drives in tank (RAID-Z)
 
I would recommend getting a LSI 1068E based controller, which comes in many flavors. Some ship by default in IR (RAID) firmware, those you would need to flash to IT firmware, making them act as normal SAS/SATA controller without RAID; perfect for ZFS.

This controller should work in Linux, Solaris and FreeBSD. On Linux and Solaris, you may need to disable MSI interrupts. On Linux/BSD you need to deactivate smartd daemon from smartmontools package. Other than that this controller runs perfectly stable. I'm using two of them on FreeBSD.

SuperMicro USAS-L8i (has non-ATX bracket; may need to remove the bracket to make it fit)
Intel SASUC8i (ATX bracket but needs firmware flash to IT-firmware)

Other brands use this controller chip as well. Specifically SuperMicro AOC-SASLP-MV8 works much worse, as it uses Marvell chip instead of LSI chip which has lot's of issues and i recommend to avoid this card if you can.

You don't need to use a separate mirrored root; kind of wasteful for just a dedicated NAS. Consider using a USB stick or Root-on-ZFS, booting straight from your ZFS pool. This also leaves you with more SATA/SAS ports for data disks.

Also consider running FreeBSD, which has better hardware support. My ZFSguru project may also be useful, but not yet matured enough. Still worth a look, especially with regard to the Root-on-ZFS option which takes only a few minutes to setup.

Are you going to use disks with 4K physical sectors? How much RAM your system has?
 
You must log in or register to reply here.
Back
Top