Upgrade/Replacement time

[Just_Visiting]

So I built my first esxi box with the following:

Supermicro X9SCM-F-O
E3-1230
LSI 9260-4i connected to Chenbro CK23601 expander
Norco 4224 Case
24x1TB HD's
16GB RAM
2x240GB SSD for VM's

It's been great, but I feel that I made some mistakes as far as going with the AIO setup. I used a windows VM as my storage pool for media and traditional data. My VM's have outgrown this server. I'm interested in seeing what's the next motherboard/processor combo. Right now I'm using 90+% on processor and RAM.

Ultimately I'd like to go with a dedicated FreeNas box. The existing motherboard, processor, RAM, and SSD's from the esxi box will be used for the FreeNas box. I'm interested in a new motherboard that supports dual LGA1356 or LGA2011. That will give me all the horsepower that I need. I have 7x3TB disks sitting here to start with for storage. The 1TB's will be retired or thrown into another box as a monthly backup option (maybe a 2nd freenas box). I have two Supermicro 836's that I can use for the freenas build along with a Supermicro 825TQ from my first fileserver build that I'd like to use for the new esxi build.

All of this is for accessing my media and data. Current VM's:

appserv-1: Running Sabnzbd, Couch Potato, Sickbeard (windows 2008 R2)
appserv-2: Running Blue Iris, Ubiquiti AP Controller (windows 2008 r2)
pbx: Running PBX In A Flash
Mediaserv: Running Plex, Subsonic (ubuntu 12 server)
Data: Storage volumes (windows 2012)

For the new esxi box, I plan to start with either 32GB or 64GB RAM. Should I install the 240GB SSDs local or in the freenas box and presented as either NFS or iSCSI?

 

[_Gea]

AIO offers up to several GB/s on internal transfers. With a dedicated box, you need a 10 Gb network.
I would first try to optimize the AIO as it should be powerfull enough

- update RAM to 32 GB
- move the storage part to ZFS software raid (best without expander)
- use HBA controller like IBM 1015 (flash to LSI9211-IT) or LSI 9207 with vt-d pass-through
- opt. replace the 1 TB disks with less, newer and faster 4 TB ones

- use NFS as shared storage (you can use BSD but Solaris bases systems like OmniOS are mostly faster with NFS)
- use a dedicated ZIL for secure sync write (ex Intel S3700-100)
For home you may partition the disk to be a ZIL + ARC
- Overprovision SSDs (10-20% with a HPA) to keep write values high
- opt. more SSDs if you need more capacity for VMs (create a ZFS Z1/2)
- give your SAN OS at least 8GB

If you build a dedicated ZFS NAS do it similar.
You may also build two AIO for failover/backup as the CPU load of the storage VM is minimal (mostly requires RAM as readcache for performance)
For a new board, check http://www.supermicro.nl/products/motherboard/Xeon/C600/X9SRH-7TF.cfm

see my howto about such a config
http://www.napp-it.org/doc/downloads/napp-in-one.pdf

 

[bds1904]

Is there a reason you are considering running freenas over omnios+napp-it?

Iirc freebsd doesn't have support for any cheap 10gb adapters atm while Opensolaris supports the brocade 1020 adapter well, and so does esxi. You could have 10gb between the 2 boxes for $120. On top of that the 1020 is a converged network adapter, meaning you could use FCOE (comstar targets on napp-it). I have always seen better latency when using FC or FCOE than when using NFS.