Your home ESX server lab hardware specs?

[peanuthead]

What is the power draw on them?

 

[s0lid]

No clue really, could buy a kill-a-watt and hook to to my ups.
Little funny detail, my ups is too beefy, it cannot calculate to VA or Watt usage of my rack, I'm not using enough power

 

[QHalo]

What fan swaps did you do to those? The ones on the STH forums?

 

[Benzino]

Updated my ESXi box. Finally got a RAID card in. Enjoying the performance bump already. Not SSD level, but still better than the single disk I tried to run way too much on.

*Gigabyte 990FXA-UD3
*AMD FX 8320
*32GB G-Skill RAM
*Dell PERC 5/i with 3 x 1 TB Western Digital Black disks in RAID 0
*Mushkin 60GB SSD running my vCenter Server virtual appliance
*MSI R5450 video card with pass-through goodness
*Corsair CX750 PSU (non-modular PSU = fail, will know for the next time)
*Corsair Carbide 200R case

 

[Kerbysj]

My ESXI server is now running. This is my first ESXI build after starting to use it at work and was blown away by how impressive it was!

Intel i7 4770
Gigabyte GA-Z87X-UD3H
32GB Corsair DDR3 Vengeance Low Profile Jet Black, PC3-12800
Supermicro SASLP-MV8 running passthrough to a server 2012 VM with

3x 4tb Seagate drive spanned
2x 3tb seagate + 2x 2tb Drive spanned

a 120gb agility 3 (not yet migrated over) for a datastore.
1tb Seagate for a datastore.
also a temp 250gb as a datastore. (will use this for iso's)

Currently i have 2 VM's running

1 win 7 box that will proberbly run as a plex server.
1 Win Server standard 2012, This runs my passthrough of my supermicro card.

Currently im using drive spanning to give me the larger drive. i know this is a big no no but its only a data store for my blurays and tv shows.

I have a microserver im going to setup with 4-5 2tb drives and run that as a weekly backup nas.


going to get a friend or two to add a couple of machines on there to stress test it before i put too much trust into it!

 

[moffeloffen]

Hi!

Have someone build a system based on haswell cpu and z87 motherboards?
I need passtrouth and and support for integrated nic etc.

 

[peanuthead]

Hi!

Have someone build a system based on haswell cpu and z87 motherboards?
I need passtrouth and and support for integrated nic etc.

You don't need a Z87 MB for passthrough, etc. I doubt many have since Haswell is so new and ESXi 5.1 was out before then.

 

[Deleted member 82943]

I think the Q series chipset might be a better fit for virtualisation

 

[_d3_]

Hi!

Have someone build a system based on haswell cpu and z87 motherboards?
I need passtrouth and and support for integrated nic etc.

Some z87 boards support passthrough, maybe all of them, but i found only some examples of asrock and gigabyte boards. asus has vt-d option in their bioses, but they claim thats only vt-x, and they cant tell you if its working or not, if that option will be there at next bios update etc.. on the other side, asrock openly supports vt-d on their boards.

On many z87 you can find intel i217v nic, for which official vmware driver doesnt exist. Unofficial driver can be found here > http://www.vm-help.com/forum/viewtopic.php?f=24&t=4391 , but nobody reported it is/isnt working yet. As alternative, buy an intel pro ct adapter, around 20-30€ new or dual port intel pro pt (ct = 1xpcie, pt = 4xpcie) from ebay.

A change in z87 from z77 is, that onboard sata controler is not passthrough capable.

 

[Vader]

I collapsed my lab into something more quiet and reasonable to manage:



3 x Servers (1 to be built)
2 Whitebox vSphere 5.1 U1
Habby Server Chassis
Corsair 600Watt PS
Supermicro Dual Skt 1366 Intel 5520 Chipset Dedicated iPMI LAN
Dual Intel Xeon 5630's
48GB DDR3 1333 ECC w/HS
Dual Intel DP 1Gb PCI-E Nics
Intel x520 CNA 10Gb (vMotion and other 10Gbe testing
Cosair 3 180GB SSD's (Host Cache for now)
Corsair 4GB USB Keys for ESXi

To be built..next project:
Dedicated storage for Private Cloud (vCloud)
Nexenta CE w/VAAI Offload Plugin for NFS
Coolermaster HAF XB
Corsair PS not sure yet
Supermicro X8SI6-F (Already Have) (Onboard LSI SAS6GB 2008)
Intel Xeon X3430 (Already Have)
Kingston 32GB DDR3 ECC 1333 RDIMM(Already Have)
Storage: 2 x small SSD for Mirrored ZIL
1 x Gskill SSD for L2ARC(Already Have)
5 x Seagate Hybrid 64MB Cache 1TB in RAID-Z2
Not sure on boot drives

 

[Jungl3]

A change in z87 from z77 is, that onboard sata controler is not passthrough capable.

Are you absolutely sure about this?

I just started reading about virtualization and had a Haswell build planned, but buying a controller wasn't in my immediate plans (and budget). I was relying in the passthrough of one of the onboard sata controllers, as i've read this was possible in some Ivy Bridge builds.

This will make me rethink about this all over again

 

[Deleted member 82943]

I collapsed my lab into something more quiet and reasonable to manage:



3 x Servers (1 to be built)
2 Whitebox vSphere 5.1 U1
Habby Server Chassis
Corsair 600Watt PS
Supermicro Dual Skt 1366 Intel 5520 Chipset Dedicated iPMI LAN
Dual Intel Xeon 5630's
48GB DDR3 1333 ECC w/HS
Dual Intel DP 1Gb PCI-E Nics
Intel x520 CNA 10Gb (vMotion and other 10Gbe testing
Cosair 3 180GB SSD's (Host Cache for now)
Corsair 4GB USB Keys for ESXi

To be built..next project:
Dedicated storage for Private Cloud (vCloud)
Nexenta CE w/VAAI Offload Plugin for NFS
Coolermaster HAF XB
Corsair PS not sure yet
Supermicro X8SI6-F (Already Have) (Onboard LSI SAS6GB 2008)
Intel Xeon X3430 (Already Have)
Kingston 32GB DDR3 ECC 1333 RDIMM
Storage: 2 x small SSD for Mirrored ZIL
1 x Gskill SSD for L2ARC
5 x Seagate Hybrid 64MB Cache 1TB in RAID-Z2
Not sure on boot drives

Wow!

 

[Orddie]

I collapsed my lab into something more quiet and reasonable to manage:

pics of what it used to be?

 

[_d3_]

Are you absolutely sure about this?

I just started reading about virtualization and had a Haswell build planned, but buying a controller wasn't in my immediate plans (and budget). I was relying in the passthrough of one of the onboard sata controllers, as i've read this was possible in some Ivy Bridge builds.

This will make me rethink about this all over again

Look at the pics yourself

gigabyte z87 board:
http://www.ivobeerens.nl/wp-content/uploads/2013/06/image91.png

other gigabyte z87 board:
http://shell.peach.ne.jp/~aoyama/wordpress/wp-content/uploads/2013/06/Z87X-UD5H-2.png

asrock z87 board:
http://blog.nogi.net/vt-d-passthrough-with-asrock-z87m-pro4/esxi-device-passthrough/

Sata controller is missing.
So a) buy raid controller or b) run local RDM.

 

[Jungl3]

Thanks, i seen those before but i guess i should have paid more attention

Back to square one

Options:
a) Go the AMD or Ivy Bridge route;
b) Start looking at XenServer;
c) Keep reading lol

All i want is to play around with virtualization a bit and collapse my media server / download station / surveillance cameras system / plex / etc, all in one box. Nothing fancy.

My main concern about AMD is power consumption and heat (i want my drives to stay as healthy as possible), but since my server won't be on 24/7 anyway, i guess that won't kill me.

And now XenServer caught my attention as well.

Any pointers anyone?

 

[peanuthead]

Thanks, i seen those before but i guess i should have paid more attention

Back to square one

Options:
a) Go the AMD or Ivy Bridge route;
b) Start looking at XenServer;
c) Keep reading lol

All i want is to play around with virtualization a bit and collapse my media server / download station / surveillance cameras system / plex / etc, all in one box. Nothing fancy.

My main concern about AMD is power consumption and heat (i want my drives to stay as healthy as possible), but since my server won't be on 24/7 anyway, i guess that won't kill me.

And now XenServer caught my attention as well.

Any pointers anyone?

What is your budget?

 

[Deleted member 82943]

Gpu pass through will be your biggest issue but going with xen makes things easier

 

[Jungl3]

What is your budget?

I don't have a strict budget really but, Hard Drives and GPU aside, i should be looking at around € 700-800 max.

Unfortunately i'm in Europe and computer parts are significantly more expensive here.

Gpu pass through will be your biggest issue but going with xen makes things easier

I don't want to go way offtopic here but, is there any reason/advantage to choose XenServer over ESXi (besides the GPU passthrough)? I know nothing about XenServer (or ESXi for that matter lol) but the little i've read about XenServer seems all advantages so far.

 

[Vader]

Awaiting the arrival of the additional components for the Nexenta build. Here is what I have so far:

 

[Jay_2]

zzzzzzz

 

[WldWzl]

and another one on the way

Would you mind posting full specs? Looking at getting one of these and wondering how you loaded it out.

Thanks.

 

[Shadowhaxor]

and another one on the way

I'm more curious on the performance, judging from how many VM's are on the box and it's resources.

 

[Jay_2]

Performance is surprisingly very good I do have 16GB of RAM and a 256GB SSD in it though and the NFS is a 4 disk RAID 5 array . The NIC is a pci-e 2 port Intel gigabit NIC.

I would say the VM that hits it hardest is the Exchange 2013 box but its on the SSD so read and writes are pretty rapid.

The last time I checked my 1810, this server, my temp monitor and my NAS was less that 100w idle.

 

[rdmustang1]

Norco 4224 Case
ASUS KCMA-D8 ATX Server Motherboard
2x AMD Opteron 4180 6core @ 2.6Ghz
128GB Registered ECC DDR3
Areca 1880i-8 RAID controller with HP SAS Expander
12x 160GB SATA HDD in RAID6
8x 1.5TB SATA HDD in RAID6
8x Intel gigabit NICs

Running 25 live VMs, and 4 as-needed, special purpose VMs on 5 virtual networks, ESXi 5.1

Total cost at time of build ~$1000

 

[Deleted member 82943]

Drool at the amount of ram. Pics?

 

[msimpson]

I'm currently in the process of gathering parts for my ESXi host, will either be 5.0 or 5.1.

Case: Cooler Master HAF 912 - just picked it up today at Micro Center, nice case for the money!
Mobo: Gigabyte 970a-ud3 Rev. 3 - bought this a few weeks ago
CPU: Haven't bought yet, but it will be a AMD FX-8350
RAM: Haven't bought yet, but it will be 2x8gb sticks(adding another 2 sticks later for a total of 32)
GPU: Haven't decided yet, need to do more research on what will pass through to a VM
PSU: Corsair 500w that's in my current PC, will buy another cheap power supply to go in my old PC
HDD: Probably a few 2TB 7200's

I also have a Dell Studio 540, q6600, 6gb RAM. Thinking about turning this into a NAS. Not sure yet. It's currently running ESXi 5.1 with a WHS2011 VM.

I would like to be able to dual boot the new ESXi host in case I ever feel like doing any type of gaming or anything. My wife plays WOW, so I'm sure she would love to game on this beast every chance she gets.

 

[trick0502]

Norco 4224 Case
ASUS KCMA-D8 ATX Server Motherboard
2x AMD Opteron 4180 6core @ 2.6Ghz
128GB Registered ECC DDR3
Areca 1880i-8 RAID controller with HP SAS Expander
12x 160GB SATA HDD in RAID6
8x 1.5TB SATA HDD in RAID6
8x Intel gigabit NICs

Running 25 live VMs, and 4 as-needed, special purpose VMs on 5 virtual networks, ESXi 5.1

Total cost at time of build ~$1000

How did you get this system for $1,000?

 

[peanuthead]

I'd imagine that some of the parts they had, were given to them and/or acquired parts over a time period to keep costs low.

 

[rdmustang1]

How did you get this system for $1,000?

Some amazing buys. The RAM was the best buy. Found a guy on eBay selling 16GB sticks for $120 each. I offered $35 each and he said yes. Mobo was open box, case was on sale, CPUs were bought on eBay for $14 each.

I'd imagine that some of the parts they had, were given to them and/or acquired parts over a time period to keep costs low.

No, absolutely everything was purchased.

 

[peanuthead]

Very cool. Mine was purchased the same way, slowly and good prices. Patience was golden this time.

 

[Deleted member 82943]

35 bucks for 16GB sticks wtf that's a steal

 

[rdmustang1]

35 bucks for 16GB sticks wtf that's a steal

Yes, Crucial 16GB DDR3 Registered ECC, new, sealed in package. I really only wanted 64gb but they were so cheap I couldn't pass the other 4 up.

 

[Eulogy]

I'm pretty happy with this:

I still have a lot of config to do on the C6100, but that's basically it for hardware for now.

 

[Benzino]

I'm pretty happy with this:

I still have a lot of config to do on the C6100, but that's basically it for hardware for now.

Nice. What is your storage configuration, and how can you possibly be using so much already?

 

[Jay_2]

If its NFS to shared storage it could be other files

 

[Orddie]

I'm pretty happy with this:
I still have a lot of config to do on the C6100, but that's basically it for hardware for now.

and this is your home LAB??

 

[Eulogy]

Nice. What is your storage configuration, and how can you possibly be using so much already?

It's NFS....

If its NFS to shared storage it could be other files

^^

and this is your home LAB??

Yup. Not as awesome as some other home labs, but it's a start.

 

[TheFriskyPen]

Here's my lab. I picked up come C1100's off of ebay for pretty cheap. Each box has 72gb of ram. Hooked it up to my ZFS box and called it a day

Still installing vm's n stuff

 

[runt]

 

[jabbernotty]

I've been looking at what hardware I should get for a home lab with the free version of vSphere. This would be my first dedicated machine, I've been using VirtualBox on my workstation so far, not a good experience. I'm using VMs for software development, I'm not expecting to need hardware passthrough other than USB. Both the vSphere and the VM storage would be on the SSD.

This is the build I'm thinking about:
- AMD FX-8350 Black Edition, 8-core clocked at 4GHz
- Kingston KVR1333D3E9SK2/16G, with ECC
- Crucial M500 240GB
- ASRock 970 Pro3 R2.0

Does that make sense, or are there any red flags?
Thanks in advance!