• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ESXi Haswell build - Help

F

fmoreira86

n00b
Joined
Aug 6, 2013
Messages
6
What do You think about this build?

CPU i5 4570
MOBO GigaByte GA-Z87M-D3H
Case. NOX NX-1 Evo
PSU LC Power 550W 6550GP V2.2 120mm
RAM 2 x Corsair Vengeance 8GB DDR3 1600 PC3-12800
HDD 2 x Seagate 1TB Barracuda 7200rpm 64MB SATA 6.0Gb/s
Cooler ARTIC COOLING FREEZER 7 PRO REV.2

I would like to have a system that is the most compatible possible to host arround 7 vm.

Thanks
 
What are the VM's doing? If they have anything intensive, and for future, I would go with 32gb ram.
 
1 webserver (500 visitors/day)
1 elastix pbx
1 active directory (study purposes)
1 exchange (study)
1 general windows server 2012 (study)

As You can see I am going to buy 2 barracuda HD but since there is no support to the onboard raid I'm considering buying just 1 SSD Samsung Pro 250GB.

I'm also thinking about do not deploy ESXi because of the lack of "home hardware" support.

Maybe I am better served with Proxmox...

What do You think guys?
 
I would go for a SuperMicro motherboard, and a 1200 v3 series cpu. You'll get better support in ESXi (atleast some realtek nics are unsupported), IPMI and ECC ram. The price is pretty much the same last time i looked.

If you want to save money, there are amazing deals on ebay for used servers (if noise isn't an issue).

I've come to prefer Proxmox over ESXi recently, but Windows Hyper-V 2012 R2 looks pretty tempting also, once it comes out.
 
Thank you.

Noise IS an issue :)

What do you think about getting an SSD (Samsung Pro 256GB) instead of 2 1TB Barracuda?

It was my intent to Raid the two barracuda (RAID 0)... but since I would need a Raid controller it would be somewhat expensive.
 
JanRK said:
Yeah, definitely go for SSDs, if you can settle for the space.

This motherboard have a build-in raid controller, at almost the same price of the standalone raid controller:
http://www.servethehome.com/Server-...erboard-review-haswell-intel-xeon-e3-1200-v3/
Click to expand...

Yes but I would prefer a "home grade" mobo. If someday I decide to use this config as a workstation....

I believe that between Raid 0 Barracuda 1TB and 1 single SSD Samsung Pro I get better performance whith the SSD. Am I wrong?

Thank you
 
You'll get better performance on the SSD. As long as you thin-provision your disks you'll be fine on storage.
 
I've also had very good luck with SuperMicro and Xeon E3 CPU.

Might want to also go with 32GB EEC RAM - if you're getting the E3.

Rootwyrm has some good info' on an ESX setup.

I followed his recommendations and built a dedicated home-ESX server on the Supermicro X9SCM-IIF-O with a Xeon E3-1270 V2 (no graphics).

I prefer the LSI RAID controller with caching though - the most expensive part of this machine. I setup a couple of RAID 0 with 4 Raptor drives. Performance is good. I know RAID0 isn't the most robust configuration... but SSD also have their issues. And, with my 4 Raptor drives there's more than enough space for several guests.

I also added an extra Intel NIC, dual port. In the end I'll probably have 6 network ports... testing out various load-balancer (e.g. F5/LTM).

I also just received (today) a new SuperMicro MB - though this one is slated for desktop/workstation. Going with the new E3-1275V3 (with graphics) on a SUPERMICRO MBD-X10SAE-O. We'll see how well CentOS v6.3 handles the built in graphics. I have a FirePro card if needed... though scavenging it from another workstation.

I'll try to post something on my blog about this new build (if sufficient rest and time permits). Here's some pictures/specs of my last ESX server.

NOTE: I'm very likely to avoid RAID0 for my next workstation - though sticking with the WD Raptors with sufficient cache on the OS. I already burned up one SSD (tend to only use SSD on laptops).
 
Go for 32GB of ram. Trust me you will need it! Exchange services won't even start without X amount of ram, depending on the version.

Also, bump the proc to the 4770 i7 for the extra threads.
Seven VMs on the i5 will be struggling to get CPU time.

And as others said here, the SSD is a must. A spinning drive will not meet the demand for more than two or three VMs. You can crush a single 10k raptor with a single vm. My Web server can crush a raid of raptors so I replaced it with a single SSD and save tons on power and cooling. If you are worried about space, get a nice samsung pro 256GB SSD for the primary drives in the VMs and get a second Samsung "non pro" 512GB SSD for the data vols in the VM or slower performance VMs. Maybe even a single spinner to hold ISO images or slow VMs that do nothing but warm the air. I always run with these three tier in my builds and I move the VMs around as I'm playing or testing.

I speak from experience here. I have several 3770k and non k builds in my garage. The Samsung 840 pro is amazing. ALWAYS go for the extra cores when building virtual hardware and always max that ram! Saving $100 will cost you several VMs worth of resources.

While I do agree with the post about going server hardware will gain you better support and maybe stability, i must say my ESXi i7 3770 builds are rock solid and host multiple domains.

Nicholas Farmer
www.pcli.me
 
Last edited:
I didn't think about this yesterday.

But what about the Z87 chipset?

If you're running ESXi... do you plan on using UEFI and overclocking?

I was also thinking about getting something with the new chipset but then held back on worries of (relatively) new drivers and increased risk of stability issues.

Gigabyte has great server and desktop boards (and so does ASRock). But, SuperMicro specializes in professional equipment - so, for me at least, it's a balance of time in working with a new player in the server space... I cannot afford the additional time required in working with less experienced support.

To be fare, I did almost purchase the Gigabyte (and have in the past). But, all of my professional equipment is built on either SuperMicro or Tyan (not so much Tyan these days).

And... let's loop back on the SSD debate?

There's absolutely no way you'll tax a pair of WD Raptors, especially in RAID 0, on a home-lab ESX server. We're not talking about production-level (enterprise) systems. We're looking at one user with a handful of test-cases. I just cannot see the SSD getting utilized to a reasonable level - at least not up to a rationalized purchase decision for SSDs.

For SSDs you're increasing costs and sacrificing storage. If you're looking at several guest OS systems, you'll need the space and not necessarily the I/O speed over 300 (roughly... average I/O ops per sec). And, you can increase those speeds with a good caching RAID card. However, a RAID card does increase your costs to something close to a pair of SSDs (that don't require a RAID to hit the high performance marks).

Behind this argument I'm also weighing in the way I setup my VM guests. I do not use a shared file-system - this isn't typical. Consequently I use up a lot more HD space than most other... enthusiasts. On the other hand, I can also quickly spin up images in very little time and ship-out (if needed).

Same goes for CPU.

My ESX server (v5.1) has about 12 guests. I seldom run more than 4 at a time. And, the CPU is rarely pegged at 100% unless I'm running aggressive tests.

RAM is another story though. I have used up close to 32GB.
 
GaryS said:
And... let's loop back on the SSD debate?

There's absolutely no way you'll tax a pair of WD Raptors, especially in RAID 0, on a home-lab ESX server. We're not talking about production-level (enterprise) systems. We're looking at one user with a handful of test-cases. I just cannot see the SSD getting utilized to a reasonable level - at least not up to a rationalized purchase decision for SSDs.

For SSDs you're increasing costs and sacrificing storage. If you're looking at several guest OS systems, you'll need the space and not necessarily the I/O speed over 300 (roughly... average I/O ops per sec). And, you can increase those speeds with a good caching RAID card. However, a RAID card does increase your costs to something close to a pair of SSDs (that don't require a RAID to hit the high performance marks).
Click to expand...
You ever try to install Exchange 2013 on a single spindle? Even if it doesn't task the disks after you get stuff installed and configured, you don't want to kill hours waiting and waiting for spindle disks or even a RAID to get moving when an SSD will fly through it.

VMware does thin-provisioning. 256GB thin-provisioned will go a long ways.

I/O does matter. A disk will get hammered the most with setup and install, and you will want the performance then.

I've installed Windows on both spindle and SSD disks in VMware, the difference is night and day.
 
nicholasfarmer said:
Go for 32GB of ram. Trust me you will need it! Exchange services won't even start without X amount of ram, depending on the version.
Click to expand...

Exchange 2013 requires 8GB of RAM if you're doing mailbox + client access on the same server. Max out the RAM.
 
benzino_86 said:
You ever try to install Exchange 2013 on a single spindle? Even if it doesn't task the disks after you get stuff installed and configured, you don't want to kill hours waiting and waiting for spindle disks or even a RAID to get moving when an SSD will fly through it.

VMware does thin-provisioning. 256GB thin-provisioned will go a long ways.

I/O does matter. A disk will get hammered the most with setup and install, and you will want the performance then.

I've installed Windows on both spindle and SSD disks in VMware, the difference is night and day.
Click to expand...

Absolutely not arguing that HDs are better than SSDs. Prefer SSD myself.
And, not arguing that an installation eats up time. But, that's a one-shot deal. So... an SSD makes things run considerably faster. But, I set the install working and simply move onto other stuff.

Most of my work is Websphere/Java-enterprise.
My WinServ (was only interested in AD) didn't 'kill me'... relatively speaking - though yes, it did take hours.

SSDs also have a shorter lifespan. Last time I checked published material on the topic was about 6 months ago. I think failure rates really spike up after about a year's worth of hours. And the more expensive/reliable SSDs are VERY pricey.

And... RAM cache is much faster than SSD. For normal use, after installation, RAM caching makes up for the 'laggy' HD speeds (only in contrast). RAM cache is on the OS guest and comes relatively pre-configured.
 
1 webserver (500 visitors/day) - 1GB RAM
1 elastix pbx (home use) - 1GB RAM
1 active directory (study purposes) - 2GB RAM
1 exchange (study) - 8GB RAM
1 general windows server 2012 (study) 2GB RAM

I'll keep the 16GB config and save some bucks,

I'll buy more RAM later this year or next year.

Since I would have to buy a RAID controller etc etc etc to get a decent performance... I'll buy the Samsung PRO SSD.


Final config will probably be:

i5 4570
GigaByte GA-Z87-D3HP
NOX ZEN
BE QUIET! SYSTEM POWER 7 80 PLUS SILVER 500W
2X Corsair Vengeance 8GB DDR3 1600 PC3-12800
SSD SAMSUNG 256 GB SATA 3 Serie 840 PRO - Basic
 
I was also looking at the Gigabyte GA-Z87X-UD3H and UD5H (wasn't sure if it was worth the $$$ for the UD5H features)

Desktop RAM (not ECC):
G.SKILL Ripjaws X Series 32GB (4 x 8GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800).

From your list, I noticed you're starting out with 2 sticks. I read somewhere that you have better odds at avoiding RAM issues if you purchase all 4 sticks together as a "4 channel set" (odd advice given this is a 2 channel architecture). Though I've followed this advice for desktop RAM, I do not purchase "sets" for server RAM (Samsung).

What lead me away from Gigabyte was Linux support and ECC RAM.

I couldn't find any support for Linux (CentOS/RHEL). SuperMicro tests their boards and rates against several non-Win operating systems. I also wanted ECC RAM - couldn't find support for this config at Gigabyte.

I ended up with a SuperMicro MBD-X10SAE-O ATX.
Leaning towards 'risk avoidance' (cough)... I didn't want to 'mess' with the new Z87 chipset for this particular computer.

My goal is to build a CentosOS (v6.4) workstation platform and from there host Win as needed. Linux has just been way more reliable, less fragile, and supporting of better hands-on controls than Windows. I'll then 'guest' Win7 from VMWare desktop. With VM snapshots I can then more reliably undo the various, crappy Win-isms.

Ideally I'd be building another ESXi box and configure 'passthrough' on the graphics card. But, that appears to be a direction of mixed success/failure. So, for now it's going to be CentOS desktop (host) with grapware (Win) guest.
 
Quick follow-up on my recent SuperMicro build - and this is on the topic of SSD vs platter disks.

Using VM Workstation...
I noticed some amazing HD performance after setup of a base CentOS 6.4 host and Win7 guest. The Win7 Guest (using HD Tune) reported an average speed of 687MB/s. Assuming, that this is mostly due to CentOS taking full advantage of free RAM (caching) and then delivering this performance to the Win7 guest.

NOTE: I did not use any RAID setup with this box

Box configuration:
1 - 1GB WD Raptor (10,000 RPM)
2 - 2GB WD "standard" (7,2 RPM)
32GB RAM (ECC)

I added most of the drive space into one main LVM. Some space was broken out for "/home" and "boot". So, I don't really know exactly which drive is in-play within the LVMs... All I know is that most of the available space is first consumed by the faster Raptor before giving it up to the slower "standard" WD drives.

So... an average 687.9MB/s speed is GOOD! (read test)
There's a wide margin though with Max @ 1313 MB/s and MIN @ 71.4 MB/s
Access time: 2.34MS
Burst Rate: 426.3 MB/s

Noting that this is measured from a Win7 guest inside VMWorkation running on CentOS 6.4
 
I would definitely get more memory if I were you. That is going to be your primary bottleneck.
 
You must log in or register to reply here.
Back
Top