New build - ESXi Home Server Lab

rufik

Limp Gawd
Joined
Sep 16, 2013
Messages
151
Hello


Im preparing to build a new home vmware lab.

I was thinking about desptop part (i7) to build that lab, but i think better hoice will be
on intel xeon processor and board also from intel (supermicro)

For that project i have at this moment:

1. Computer case from Fractal Design DEFINE XL R2 TITANIUM GREY
http://www.fractal-design.com/?view=product&category=2&prod=112

2. Hardware RAid controller from LSI
LSI MegaRAID SAS 9271-8i

3. Six hard drive's from WD (6 x 3 TB)
WD Red WD30EFRX (3TB, 64MB, SATA/600)

4. RAM 16 GB / soon upgrade max to 32 GB
G.SKILL DDR3 2x8GB 1600MHz CL10 RipjawsX

And right know i need

1. CPU
2. BOARD

-> For Board from INTEL <-

http://ark.intel.com/compare/66137,71384,71385,61042

Which one will be best choice ?

-> For processor (CPU) <-

http://ark.intel.com/compare/75054,75465,75057,75464,75056

Which one will be best choice ?

Most important difference is SOCKET for LGA1150 (Socket B2 LGA-1356 vs LGA1150) and size of board (SSI ATX 12" X 9.6" vs uATX)

Socket B2 LGA-1356 vs LGA1150 ?
SSI ATX 12" X 9.6" vs uATX ?

ps.

What are most difference's beetwen processor Intel Xeon E3 and E5 ?
 
Before we go anywhere, what are you using the system for? What kind of load on the vms will you place on them?
 
Before we go anywhere, what are you using the system for? What kind of load on the vms will you place on them?



Hey

Sure :)

1. Active Directory - Windows Server 2012
2. Linux VATTA - Internet connection
3. Linux for Cisco, Juniper lab (ios, jun os, asa)
4. Free NAS
5. Linux for DNS
6. Windows 2003 srv to connection from VPN (terminal)
7. Linux web server with mysql
8. Linux services monitoring (centreon, cacti)
9. ESXi 60 day evo... lab
 
Are you planning to use FreeNAS As your main storage - All in one solution ? If so you should go with HBA not hardware Raid card.
Rest - like E5 vs E3 is related to fact if you want for example more than 32gb ram in the future or how many pcie lanes do you want.
 
Are you planning to use FreeNAS As your main storage - All in one solution ? If so you should go with HBA not hardware Raid card.
Rest - like E5 vs E3 is related to fact if you want for example more than 32gb ram in the future or how many pcie lanes do you want.

Hello

FreeNAS only for shared disk allocation for some VM's

Main storage will be provided by Active Directory (into domain for home users)

I want use max 32gb memory.

Also i have question

http://ark.intel.com/compare/75054,75465,75057,75464,75056

Can i use 32 gb memory in my board if on specification i can see

Max Memory Bandwidth - 25.6 GB/s

What it mean's ? Because i don't understand that

I will have 32 GB and Max Memory Bandwidth 25.6 GB/s.

Do there will be system slow perfomance or what ?
 
Don't get hung up on the max. memory bandwidth. For where you are at, just ignore it.
 
Don't get hung up on the max. memory bandwidth. For where you are at, just ignore it.

Regarding you previous question about pciex

1. controller raid
2. i need one for adidtional NIC

and i thik thats all

I have here integrated GPU so i dont need externel GPU.

Do i need more pciex ?

ps. Could you explain for newbie what is going with that memory bandwith, what is a reason of this value ? etc... ?
 
Supermicro (for Xeon) vs other

I've had very good luck with SuperMicro.
I have two computers with Xeon E3 CPUs. One currently replacing a troubled workstation. The other dedicated for ESXi. Would have preferred the E5... but they're expensive and I've noticed the CPU generally underutilized on my 2xCPU workstation. So, couldn't justify the extra budget for underutilized horsepower.

The new E3 (haswell?) also runs cool and quite. Doesn't heat up the office.

Some things to think about:
- RAM gets used up very quickly... 32GB, in my opinion, is bare minimum.
- Get lots of efficient storage. I spin up several VMs for quick testing, analysis, and then move them to cold storage. I don't have the time to move them off to external storage. It's way easier to simply move them internally, within the box, or just leave them be (don't forget to take notes on which-did-what). When the disks fill up? Buy more. Disks are cheep.
- Make sure you have enough PCI-E slots for extra NICs (intel 2xport). The additional ports are a nice feature if you like routing traffic through load-balancers (e.g. F5), etc.
- IPMI is a "must have". Couldn't live without it.
- LSI Raid. Though some here disagree... I like the caching controller card. It's damn fast with the 512MB (or more). I didn't get the on-card battery backup. Instead I have my machine plugged into a big UPS.
- Expensive SSD? I personally consider SSD overkill for a home lab "server" (ESXi). Many here disagree.
 
IPMI is like an onboard KVM (keyboard Video Mouse).

IPMI server runs on the motherboard, uses its own network/lan port, and connects to a computer running the Java IPMI client.

For example, my SuperMicro motherboard has 3 network ports:
1 and 2 are basic on-board NiCs
The 3rd is the IPMI port.

I can then connect to the server via my desktop running the IPMI client software.
The client presents a view just like a display plugged directly into the computer. You even get to see boot-up and can interrupt for BIOS configs, etc. (however... be careful about flashing the BIOS or IPMI software over IPMI).

Some features:
  • start/stop
  • temp monitors
  • fan speeds
  • remote media

Link to SuperMicro IPMI page
 
IPMI is like an onboard KVM (keyboard Video Mouse).

IPMI server runs on the motherboard, uses its own network/lan port, and connects to a computer running the Java IPMI client.

For example, my SuperMicro motherboard has 3 network ports:
1 and 2 are basic on-board NiCs
The 3rd is the IPMI port.

I can then connect to the server via my desktop running the IPMI client software.
The client presents a view just like a display plugged directly into the computer. You even get to see boot-up and can interrupt for BIOS configs, etc. (however... be careful about flashing the BIOS or IPMI software over IPMI).

Some features:
  • start/stop
  • temp monitors
  • fan speeds
  • remote media

Link to SuperMicro IPMI page

Nice. I checked it i want it ! :)
I can see that on intel boards all of them have ipmi.

I am almost decided for E3 Intel XEON board for chipset C226

In my opinion C226 is better. Am i wrong ?
http://ark.intel.com/compare/63984,36785,64015,75522,75516

But there is a problem regarding Supermicro board, only one is supporting ipmi :((((

http://www.supermicro.com.tw/products/motherboard/Xeon/C220/X10SLH-F.cfm
Big minus this is mini ATX. Better for me will be ATX.

Full ATX has more pciex, mini not ...
Supermicro has two full ATX boards
http://www.supermicro.com.tw/products/motherboard/Xeon/C220/X10SAE.cfm
http://www.supermicro.com.tw/products/motherboard/Xeon/C220/X10SAT.cfm



Im confused and i need your advise

Main questions

1. uATX vs ATX ?
2. Board with IPMI or SuperDoctor® III (for Supermicro) ?
 
Recommend reading this blog:
Rootwyrm's Corner
and
BabyDragon II Hardware Compatibility List


re:
Main questions
1. uATX vs ATX ?
2. Board with IPMI or SuperDoctor® III (for Supermicro

uATX vs ATX: I would go with function and features first. Then, begin looking at size issues.

How I did this:
- watch for the "extended" description in mobo size - example: "e-atx". The "extended" dimensions aren't all the same. Absolutely not an issue as long as you have a big case and are not shy of slight modifications. Most people blog about modifying cases to fit the larger SuperMicro boards.

I would avoid the "proprietary" sizes as these are specific to SuperMicro cases. If you're going proprietary then you're better off buying the box as a unit (don't mess with installation).

My last two SuperMicro boards were uATX dimensions (NOTE: and what is the "u" in uATX? Honestly do not know. They did fit my LianLi ATX mid-tower case).

IPMI vs Doctor

Haven't used "SuperDoctor"...
I do use IPMI - it works well from a win7 Desktop.
NOTE: I haven't been able to find a VMWare vSphere Client for Linux (CentOS). Consequently, I'm currently running a Win7 guest on my CentOS desktop.

Xeon CPU:
If I had more money and time... i'd go with the E5.
However, the E3 is a very good workhorse. No issues. Very popular and well supported.
Make sure you get an E3 with hyper-threading! (review the E3 models carefully)
Also, unless you need the embedded video, don't pay any extra. Just happened that I purchased my workstation E3 CPU on special so the extra Graphics GPU (embedded) was a "freebee". However, couldn't get the Xeon graphics working for a 3x display desktop and eventually switched over to FirePro (I have a CentOS workstation).

My ESXi server has the E3-1270 V2 3.5GHz model (purchased this prior to V3).
My Workstation has the newer V3 addition... but, "meh..." no real empirical differences given my applications.

Here's my ESXi server specs (please note the similarities to rootwyrm specifications)

Haven't had a free minute to write-up my new E3 workstation build...

Motherboard:

Carefully review support and compatibility for RAM (ECC), CPU, multi-threading, and passthrough. SuperMicro does an outstanding job of listing what their boards can and cannot do. They also list tested OS (ESXi, etc.)

Don't think you're going to get IPMI support on the "cross-over" models for workstation (extra PCI-E 3.0 x16). If your server is doubling as a workstation... then you really don't need IPMI (you'll have a physical display connected anyway). But, if you're building an ESXi server... then you don't need the extra x16 PCI-E slots unless you're adding some damn-expensive RAID cards... etc. This then brings up a point on exactly what sort of adapters/cards you intend to add to your server.

Make sure you have enough slots on your motherboard. PCI-E will spread (the x16 becomes an x8, etc) with additional cards. Read the motherboards documentation and understand EXACTLY what each slot supports based on the usage and number of slots consumed.
 
Recommend reading this blog:
Rootwyrm's Corner
and
BabyDragon II Hardware Compatibility List


re:


uATX vs ATX: I would go with function and features first. Then, begin looking at size issues.

How I did this:
- watch for the "extended" description in mobo size - example: "e-atx". The "extended" dimensions aren't all the same. Absolutely not an issue as long as you have a big case and are not shy of slight modifications. Most people blog about modifying cases to fit the larger SuperMicro boards.

I would avoid the "proprietary" sizes as these are specific to SuperMicro cases. If you're going proprietary then you're better off buying the box as a unit (don't mess with installation).

My last two SuperMicro boards were uATX dimensions (NOTE: and what is the "u" in uATX? Honestly do not know. They did fit my LianLi ATX mid-tower case).

IPMI vs Doctor

Haven't used "SuperDoctor"...
I do use IPMI - it works well from a win7 Desktop.
NOTE: I haven't been able to find a VMWare vSphere Client for Linux (CentOS). Consequently, I'm currently running a Win7 guest on my CentOS desktop.

Xeon CPU:
If I had more money and time... i'd go with the E5.
However, the E3 is a very good workhorse. No issues. Very popular and well supported.
Make sure you get an E3 with hyper-threading! (review the E3 models carefully)
Also, unless you need the embedded video, don't pay any extra. Just happened that I purchased my workstation E3 CPU on special so the extra Graphics GPU (embedded) was a "freebee". However, couldn't get the Xeon graphics working for a 3x display desktop and eventually switched over to FirePro (I have a CentOS workstation).

My ESXi server has the E3-1270 V2 3.5GHz model (purchased this prior to V3).
My Workstation has the newer V3 addition... but, "meh..." no real empirical differences given my applications.

Here's my ESXi server specs (please note the similarities to rootwyrm specifications)

Haven't had a free minute to write-up my new E3 workstation build...

Motherboard:

Carefully review support and compatibility for RAM (ECC), CPU, multi-threading, and passthrough. SuperMicro does an outstanding job of listing what their boards can and cannot do. They also list tested OS (ESXi, etc.)

Don't think you're going to get IPMI support on the "cross-over" models for workstation (extra PCI-E 3.0 x16). If your server is doubling as a workstation... then you really don't need IPMI (you'll have a physical display connected anyway). But, if you're building an ESXi server... then you don't need the extra x16 PCI-E slots unless you're adding some damn-expensive RAID cards... etc. This then brings up a point on exactly what sort of adapters/cards you intend to add to your server.

Make sure you have enough slots on your motherboard. PCI-E will spread (the x16 becomes an x8, etc) with additional cards. Read the motherboards documentation and understand EXACTLY what each slot supports based on the usage and number of slots consumed.

I need pcies because i have RAID LSI 9271 8i :)


What about power suply for E3 or E5 ?

600 W will be ok ? or more ?
 
It sounds fun and great to do this, but I am going to suggest something that makes way too much sense and just works well.

Get 2 HP Gen 8 MicroServers. Replace their processors with Xeon E3-1220V2 processors and put in 16GB of ECC ram. Then you have 2 really small very powerful LAB devices. Storage is of your choosing, but I prefer something like a Synology for ease of use. This lab may not be fun for those that want to just build, but it works well if not better for LAB purposes. Also it will give you the opportunity to understand why large companies stick with true production servers for their setups. They just work.

Also if you want the ILO to be available, you can do that with the purchase of an advanced license key from ebay.
 
It sounds fun and great to do this, but I am going to suggest something that makes way too much sense and just works well.

Get 2 HP Gen 8 MicroServers. Replace their processors with Xeon E3-1220V2 processors and put in 16GB of ECC ram. Then you have 2 really small very powerful LAB devices. Storage is of your choosing, but I prefer something like a Synology for ease of use. This lab may not be fun for those that want to just build, but it works well if not better for LAB purposes. Also it will give you the opportunity to understand why large companies stick with true production servers for their setups. They just work.

Also if you want the ILO to be available, you can do that with the purchase of an advanced license key from ebay.


Could you show me some URLs from shop ? ebay ?

It should be fine. 350w is more than enough. Now buy the parts! :)

I need to be sure, that we understand :)

I will install

6 x WD RED 3 TB
1 x PCIEX LSI harware raid
2 x 8 GB ECC memory

E3 or E5 procesor :)
I need to compare all of super micro for LGA2011 and signle CPU/ATX only mainboard

350W it is ok ? Im nor sure :p

So I need advice in that case
 
(NOTE: and what is the "u" in uATX? Honestly do not know. They did fit my LianLi ATX mid-tower case).

u should actually be the greek letter mu (&#956;) as in &#956;ATX. Its the scientific standard prefix for micro, hence micro atx.

I dont understand :p
Could you explain ?

To do this start looking at the datasheets/web pages of the components you want to use. They should all state maximum power draw. Add it all up and you'll see how big a power supply you will need. You will be surprised how frugal you can make a virtualisation server!

:)
 
I dont believe. Men it a Xeon TDP likr in i7. RAM, Six (6) hard drisks from WD (RED)

Wrr i need calculate :p
 
Back
Top