Building new Nexenta box, drive placement?'s


Supreme [H]ardness
Dec 22, 2002
I'm in the process of completely revamping the lab storage and moving the equipment to my bro's business as he has a clean air conditioned space I can use and dedicated 50/50 Fiber w/Business Class backup from Roadrunner. I will be reusing a lot of equipment to save on costs.

I'm working on the Bare Metal Nexenta Array and have the following:

Supermicro CSE-216A-R900LPB 24 x 2.5" Bays, SAS/SATA 6GB Backplane-Redudant 900Watt PS.
Supermicro X8SI6-F (On board LSI2008)
Intel Xeon 3440 w/Thermaltake LP HSF
4 x 8GB Kingston ECC Registered 1333
1 x IBM 1015's Flashed IT Mode
2 x Samsung 830 Pro's 256GB - ZIL
2 x Corsair Force 120GB - L2ARC
2 x Corsair Force 60GB - Onboard Controller Mirror Nexenta/OS
12 x Seagate 2.5 1TB 64MB Cache Hybrid 8GBSSD Drives
Intel Pro Dual PT/1000 Low Profile Controller (4 x 1Gb NIC's, 1 x 10/100Mb iPMI)

Looking for advice on drive placement. I'm thinking from a resiliency standpoint, I should split the SSD's across both controllers, but what will that do for performance? If it's negligible, i'd rather have the resiliency. ZIL will be mirrored, L2ARC will be Striped, Nexenta/OS will be mirrored. The remaining drives, not too sure, thinking RaidZ with Spare. Not really sure I want to go mirrored vdevs due to capacity loss.

Workloads: Will be primary storage to a couple vSphere Hosts, don't want to have to worry about performance. Will be running all core vSphere products, Nested Hyper-V and ESXi with vCloud Director/vCAC as Primary UI. I will also be running my Wordpress Site from this and some other VM's my bro will use for his security software, AP controller software and of course CORE Domain Infrastructure.

NFS and iSCSI will be the Storage Protocols. Dedicated Uplinks for both and iSCSI multipathing over 2 x HP 1810-24G switches.
It does not matter if you split your SSDs over two controllers or connect them to the same unless you have a pool layout that can work with a single controller.

L2ARC is not striped, its more a load balancing over 2 SSDs.

The Samsung are not a good ZIL, you may use them as a mirror for performance sensitive VMs. A SSD like a Intel S3700 would give you a much better ZIL Accelerator.