Yet Another ESXi FreeNAS build

M

MrMadMan

n00b
Joined
Nov 28, 2013
Messages
5
Hi!
My first post on the [H] forums, so go easy on me ;)

I'm planning on building an All-In-One and would like to hear your comments on hardware setup and configuration. Here we go!

Vision
A fairly powerful virtualising server with tons of disk space for personal (family and friends) use.

Aim
A quiet ATX case, with room for at least 6 storage drives (+ 2 for VM storage), server grade motherboard, Xeon quad core and 32GB ECC RAM.

Build
  • Motherboard: Supermicro X10SL7-F
    I'm thinking server grade Haswell here. This board have become very popular as it sports dual Intel I210-AT NICs and the LSI 2308 SAS controller.​
  • CPU: Intel Xeon E3-1230v3
    No competition price/performance wise.​
  • Storage Controller: IBM ServeRAID M1015
    This seems to be THE controller to get. The NAS storage goes here.​
  • RAM: Kingston ValueRAM DDR3 PC12800/1600MHz CL11 ECC 32GB (KVR16E11K4/32)
    Do I have to go 4x8GB or can the above mentioned motherboard handle 16GB sticks? What about chip compatibility? I'm kind of struggling with finding what RAM are compatible...​
  • Storage (HyperV): Kingston DataTraveler 100 G3 8GB
    ESXi goes here.​
  • Storage (HyperV backup before upgrade): Kingston DataTraveler 100 G3 8GB
    Clone ESXi to this one before attempting an upgrade.​
  • Storage (Fallback FreeNAS): Kingston DataTraveler 100 G3 8GB
    A fallback FreeNAS that's installed first and thoroughly tested. Can later be used if for some reason the ESXi installation goes bonkers​
  • Storage (VM:s and snaps): 2x 500GB SATA HDD
    Use on-chip LSI 2308 RAID-1 for these and store VM:s and possibly snapshots on them.​
  • Storage (Pool): 5x WD Green 2TB
    I have these in a RAID-Z configuration on my current server. Will in time upgrade to a larger RAID-Z2 or something​
  • Enclosure: Fractal Design Define R4
    Reasonably priced and fairly quiet. Takes 8 (+3) drives which allows room for upgrades​
  • PSU: Seasonic G-450 450W
    Can't go wrong with Seasonic​

Configuration
  1. Make sure the 2308 is flashed to IR mode and configure a mirror on the 2x500GB drives.
  2. Make sure the M1015 is flashed to IT mode.
  3. Export the zpools on my old setup.
  4. Install FreeNAS on one of the USB keys, import the zpools and make sure the storage is working as intended.
  5. Export the zpools again.
  6. Install ESXi on another USB key (in the onboard USB port!)
  7. Install FreeNAS as a VM and import the zpools here (last time!)
  8. Share the datasets with other clients using iSCSI or NFS (I have to read up on this)

Crash plan
If I want to upgrade ESXi, I'll make a backup to another USB key first.
If the ESXi setup totally blows up for another reason and takes the FreeNAS VM with it, I have a backup on a USB key that I have previously tested.
I have yet to figure out how to backup the other VM:s though. Maybe backing up to the NAS would be okay...

Finally
So, what do you think? Does this sound like a good setup?

What about the migration from my current setup (Debian box running zfs-on-linux)? Seems safe?
 
In regards to the RAM, Supermicro mobos tends to work well with Kingston RAM. Since 16GB sticks of RAM aren't exactly widely available, I doubt there'll be support 16GB sticks with that motherboard. Not to mention that the max RAM amount supported is 32GB.
 
Why not use the LSI2308 for your storage pool? U can flash the LSI2308 to IT-mode.
My setup is almost the same

Im planning to use a SSD to store my VM's on and I will passthrough my 6x3TB RAIDZ2 to ZFSguru using the LSI2308. A additional M1015 is not needed.
 
FREAKJAM said:
Why not use the LSI2308 for your storage pool? U can flash the LSI2308 to IT-mode.
My setup is almost the same

Im planning to use a SSD to store my VM's on and I will passthrough my 6x3TB RAIDZ2 to ZFSguru using the LSI2308. A additional M1015 is not needed.
Click to expand...

Because I want to have redundancy where I store the VM:s.
And to use the 2x500GB in RAID-1, I need one controller in IR mode.
 
Makes sense yes. Nice build!
You're the same guy from the freenas forums right? Supermicro is gonna either send me a new board or repair it after all. Looking forward to your updates!
 
Last edited:
FREAKJAM said:
Makes sense yes. Nice build!
You're the same guy from the freenas forums right? Supermicro is gonna either send me a new board or repair it after all. Looking forward to your updates!
Click to expand...

Yeah. That's me allright :)

As you might have noticed, the build has been revised slightly. I'm starting to feel very confident in the setup, so I'll probably place an order before the weekend is over.

I will make sure to update on both forums. If I create a thread asking for help on a forum, I feel obliged to report the results as well.

Thanks for showing interest in it!
 
NFS - don't use iSCSI with FreeBSD based products still. The target is broken, doesn't handle things right. :)
 
lopoetve said:
NFS - don't use iSCSI with FreeBSD based products still. The target is broken, doesn't handle things right. :)
Click to expand...

Really? I use a all-in-one with freenas and it works with iSCSI as far as i know. I did do some changes in ESXi causing it to drop conncetion while under heavy load, but that was for NFS not for iSCSI.
 
lopoetve said:
NFS - don't use iSCSI with FreeBSD based products still. The target is broken, doesn't handle things right. :)
Click to expand...

I've heard about this before. Do you have a source of this information? I'd like to do some reading up...
 
I did a lot of work internally at vmware with the BSD target and it has trouble with significant numbers of reservations over time and when it finally collapses, it results in major data corruption. If you're running only a few VMs/hosts per LUN things tend to go fine but as that number climbs things have a habit of going very poorly. Given that it is not a supported target nit much work has been done to see about alleviating the problem. :)

I'll grant that o haven't done a lot of work recently on it, but it was still broken a year ago.
 
You must log in or register to reply here.
Back
Top