Question regarding ESXi with ZFS build

X

Xfinity

n00b
Joined
Apr 14, 2009
Messages
57
Hi there!

I need some input regarding an issue I am having with my server that I am upgrading to better parts.

Here is my new hardware:
Supermicro X9SCL+-F
Intel Xeon E3-1240v2
16GB ECC DDR3
HP NC360T Network Card
2 x IBM M1015

Disks I am able to use:
8 x 2TB WD20EADS
2 x 2.5" 250GB WD2500BEKT
1 x 2.5" 250GB WD2500BEVT
2 x Intel SSD 320 80GB
2 x Intel SSD 320 40GB
1 x Kingston 8GB USB Memory Stick

I can run 2 disks either RAID 0 or 1 using this little machine:
Icy Dock MB982SPR-2S

Purpose: Run VMs with ESXi. 1 Win7 VM, 1 Win2K8 Server VM, 1 Linux VM and 1 OI VM
for ZFS.

Needless to say, I was very much inspired by the napp-it All in One server build for this upgrade ;).

But I have been working to much this week (60+ hours) so when its time to build/upgrade the box I cannot decide where I utilize the disks best. I am thinking in using the USB memory stick as a boot drive for EXSi, use the Icy Dock MB982SPR-2S with RAID 1 on the WD2500BEKT-drives for datastore but use 1 x 80GB SSD for the Win7 VM. Last but not least use 6 x WD20EADS-drives to setup a raidz2 but keep 2 as spares because of the hard drive prices (and I don't need all that space for what I am aiming at).

Is my thinking sound or can I optimize this better using another type of setup regarding the hard drives that I have at my disposal? Bring it!

Thanks!
 
Best practices with ZFS, is to avoid any raid card. ZFS can not protect against data corruption if you use raid cards. ZFS needs exclusive access to disks.

What chassis are you going to use? Are your VMs very heavily loaded? If yes, you need much IOPS, which is best achieved with mirrors. If not, you can get away with raidz2 and lot of RAM, 16GB is ok.
 
I don't think this is correct. If a raid card is presenting a logical disk to ZFS - if ZFS writes X to the logical disk and the raid card corrupts the data, how would ZFS not detect that on reading it?
 
The M1015 cards are flashed to IT mode, so no raid there for the ZFS.

Regarding the chassi it's big enough to handle what I am aiming at ;). The Linux and Win7 VM are heavily loaded.
 
danswartz said:
I don't think this is correct. If a raid card is presenting a logical disk to ZFS - if ZFS writes X to the logical disk and the raid card corrupts the data, how would ZFS not detect that on reading it?
Click to expand...
What are you trying to say, that it is perfectly good to use a hardware raid card with ZFS? Or are you talking about JBOD?
 
danswartz said:
I don't think this is correct. If a raid card is presenting a logical disk to ZFS - if ZFS writes X to the logical disk and the raid card corrupts the data, how would ZFS not detect that on reading it?
Click to expand...


ZFS should detect the corruption, but because it treats the RAID lun as a single drive, it can't correct it, as would be the case if the redundancy was applied by ZFS itself (ie mirrors or one of the RAIDZ flavours).
In such cases, you'd then have to restore the affected files from backup!

You can certainly use RAID cards/devices with ZFS, but it's perhaps not quite as straightforward as using directly attached disks (though it's nothing uber tech either TBH).
The caveats are really the self healing features (as above), and the behaviour of cache flushes!
 
Xfinity said:
Ok, how about the best disk layout for this build? ;)
Click to expand...


Best for what? ;)


You could look at

1 x 2.5" 250GB WD2500BEVT as the boot drive, off a MBD SATA port

4 x 2TB WD20EADS (or less and increase as and when needed)
1 x Intel SSD 320 80GB
1 x Intel SSD 320 40GB
off each M1015 card, then create a pool with up to 4x ZFS 2TB mirrors (each mirror using a disk off each card), 2x 80GB L2ARC and possibly 1x mirrored 40GB ZIL log (though you could partition these as ZIL logs needn't be anywhere near this big, and use the other, say 32GB on each for something else - mirrored archive storage perhaps. Debatable about mirroring ZIL logs, but that's another topic)

2 x 2.5" 250GB WD2500BEKT
perhaps in Icy dock as RAID1 for data store, as you suggested.
Personally I'd probably mirror them using a couple of MBD SATA ports though, and flog the Icydock on ebay!


That'd give you more IOPs from the ZFS pool - if you are happy with less, then go RAIDZ2 as you suggested!
 
You must log in or register to reply here.
Back
Top