Pre-ZFS Build Qs

[kronik8]

So, I have the following lined up, I think:

NORCO RPC-4020 4U Rackmount Server Chassis w/ 20 Hot-swappable SATA/SAS 6G Drive Bays

ASUS Z8PE-D12(ASMB4-IKVM) Dual LGA 1366 Intel 5520 Tylersburg SSI EEB 3.61 Dual Intel Xeon 5500 and 5600 Series w/ Remote Management Server Motherboard

CORSAIR Professional Series AX750 750W ATX12V v2.31 / EPS12V v2.92 80 PLUS GOLD Certified Modular Active PFC Power Supply

Intel Xeon E5520 Nehalem 2.26GHz LGA 1366 80W Quad-Core Server Processor BX80602E5520

Crucial 8GB (2 x 4GB) 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10600) ECC Registered Server Memory Model CT2KIT51272BB1339

The plan, at current, is to build an ESXi box, passthrough PCI to a single VM which will run either Solaris 11 Express + Napp-It or Nexenta, I'm not sure yet. From there, I plan on building a couple more VMs, maybe running AD for the house, a test machine to fart around with patches, etc.

Comments on planned architecture:

I like the idea of having OOB hence the server-grade Asus.
We have a host of SuperMicro's at work and they've been nothing but trouble so I'm avoiding them altogether, if I can help it.

Questions:

Is one vCPU (1x core) / 2GB of RAM sufficient for a ZFS VM?

Recommendations for ZIL drives?

Recommendations for ESXi kernel drives? I would think they wouldn't need to be too peppy but my gut tells me to go with RAID 1. That could be the data center in me talking.

Connections in the Norco box? I plan on starting with 4 Samsung 2TB drives so I should be good not purchasing any additional cards and using onboard SATA (there are 6 total) -> SFF8087, correct? I will be going with Intel SASUC8I when the need for expansion
arises or budget allows.

Has anyone run into bandwidth issues with only two onboard NICs? Only one NIC will be associated with VM networking traffic. This will be a home file server attached to:

1x MacBook Pro, using AFS for Time Machine backups.
1x Gaming Machine, not a large consumer, maybe video from time to time and backups.
1x HTPC, largest streaming consumer, backups.
1x Latitude, work machine, backups.
PS3/Xbox 360 streaming.

I may at some point go with a Quad NIC just to get increased bandwidth out of the box but onboards will do for now.

Am I missing anything else?

 

[sub.mesa]

Why registered memory? I would prefer 16GiB DDR3 unbuffered to 8GiB DDR3 registered.

With 16GiB DDR3, you could decicated 12GiB to ZFS and use the 4GiB for other stuff. If you do it the other way around (give ZFS only little memory and other VMs the largest part) you may encounter poor performance. The best performance you get from 2GiB RAM per HDD. So try to have the RAM : DISK ratio in check; sure you can power 30 disks with 4GiB RAM but you would waste alot of potential performance that i wouldn't want to waste.

4K sector drives love these configs:
RAID-Z: 3, 5 or 9 disks
RAID-Z2: 6 or 10 disks

ZIL/SLOG drives should carry a supercapacitor for protection; Intel G3 and Marvell C400 fall into this category. Vertex 3 Pro would also have one, but i would avoid OCZ if possible. So wait with buying an SSD until you can buy a good one with supercap and use for both SLOG and L2ARC feature, to speed up your array.

 

[apnar]

Connections in the Norco box? I plan on starting with 4 Samsung 2TB drives so I should be good not purchasing any additional cards and using onboard SATA (there are 6 total) -> SFF8087, correct? I will be going with Intel SASUC8I when the need for expansion
arises or budget allows.

The problem here is passing through ESXi to the storage VM. If you're going to use the on-board controller for the ESXi boot drives then you can't pass it through to the VM with VT-d. So you may want to consider pulling the trigger on the Intel SASUC8I or take a look at the IBM ServeRAID M1015 (it's a rebranded LSI 9240-8i with the RAID-5 features locked out) which you can find for $70-$75 on ebay. The other option would be going with a board (like some of the super micros) which has a separate SAS controller on-board that you can pass through.

 

[vipera66]

The NORCO RPC-4020 has regular SATA/SAS connectors only. You may want to consider the NORCO RPC-4220 instead, which has SFF-8087 (4 channels in 1 cable).

Also note that LGA1366 chipsets are triple channel, so you will need to install memory in sets of 3.

 

[dodgeboy]

You will definitely want to give the ZFS VM a bit more RAM. 4-6GB should be enough to start with. I gave mine 2 cores as well, because I have compression enabled. You can see it hit the kernel pretty hard when you start to throw data at it.

I used a OCZ (I know, I know...) 120GB Agility2 SSD for ZIL and L2ARC. I gave 10GB to ZIL, and the rest to L2ARC.

I doubt you will be able to get much beyond 1Gbit/sec of traffic to/from it, so I wouldn't bother with the 4 port NIC.

For reference, here is what I did:
http://hardforum.com/showpost.php?p=1036934894&postcount=1146

Dave

 

[TypeRazor]

Also note that LGA1366 chipsets are triple channel, so you will need to install memory in sets of 3.

This is partly good advice, Yes the 1366 chipset is triple channel but that does not mean you must install in sets of 3

You will be fine with 1-6 sticks of RAM, it really doesnt matter that much