• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Esxi Power Consumption

L

LJensen

n00b
Joined
Apr 22, 2012
Messages
23
Hello All

I have a small esxi white box where i would like to lower the power usage.

My setup:

Motherboard: Supermicro X9SCA-F
CPU: Intel Xeon E3-1230 (version 1, 80w max tdp)
PSU: Silverstone ST40F
Ram: 16gb Kingston ECC
RaidCard: Perc 5i (Raid 10)
Harddisk: 4 X 2.5" Western Digital Blue 320gb (WD3200BEVT)
Esxi is booted from 8gb USB drive

With 2 vm's running (server 2008 with exchange and server 2012) it uses 100-110w pr hour, which for me sounds like too much :) Looking at the vSphere client it "idles" at around 200Mhz, so i don't actually need the 4 X 3ghz it provides.

Booted the server with no raidccard and no harddisks, it then uses 48w pr hour. Does the raid+harddisks really use that much power, or is something off here?

According to WDC.com the harddisks should use 2.5w read/write and 0.85w in idle.
http://wdc.custhelp.com/app/answers/detail/search/1/a_id/5391#

I thought about changing the cpu to one with a lower TDP (E3-1220L v3), but not sure how much that would actually give, also thought about buying 1 ssd for storage and work on a better backup solution (no raid)

Please help me save some power :) Price range for upgrade = 0-1500$, the less the better :)
 
Last edited:
The numbers without the controller certainly look reasonable, even if a bit high, but considering the board is well equipped it is not a surprise. I have a similar setup with similar power consumption (i5-3570K, 32GB RAM, SSD, 80Plus Bronze 750W PSU; it uses 39W in idle).

The numbers with the controller and hard drives on other side look insane - i have 14 3.5" WD Green or Red 2/3TB drives, connected to the onboard Intel/Marvell controllers, plus to a IBM M1015 controller, and even with all that (plus a Intel dual gigabit ethernet card and a onboard wifi adapter) i still hit only 120W with drives spinning and 71W when drives are in standby.

If i were you i would try to connect temporarily the hard drives to the motherboard, pull out the Perc 5i card and try to boot the system that way and measure the power consumption. Sure, your RAID will be one for that boot, but you need to just figure out the effect the RAID card has on power consumption. It is the likely culprit unfortunately.

Disclaimer: All power consumption figures are from the socket, that means PSU plays role in the numbers; measured @ 230V we have here in Europe. Sure, i boot plain Linux instead of ESXi, but i doubt that it would have that much of an impact on idle power consumption.
 
Thanks for the reply :)
I had a gut feeling that the drives were messing up the power consumption. Might have to test the drives individually to see what is going on. I know it will mess up the raid10, but it will be worth it in the long run.
The motherboard is really nice and i love the IPMI, but i would change it, if that would lower the W alot.
What motherboard do you use?
 
I changed the power management in vSphere from Balanced to Low and it seems to drop around 10W
 
Well you technically don't even need to connect the drives, just remove the RAID controller from the motherboard. Keep the hard drives plugged in the power and technically you don't even need to disconnect the cables between the RAID controller and the drives, just have the card not plugged in the motherboard.

Boot up, check power consumption, shut down. I highly doubt it is the drives, as even the most insane drives use 10W/drive at highest possible load (continuous write with lots of seeking).

My motherboard is P8Z77-V, which is pretty similar to what you have in feature set, and as you can see, the power consumption without the drives is very similar (39 vs 48W; the difference can be attributed to different low power efficiency, hyperthreading of your CPU; less efficient power circuitry on your board).
 
The Perc 5i is likely your culprit. Scrap that and use the hard drives for non-raid backup storage if you can, an SSD will give better performance (power and IO) for active VMs if you can justify the cost per your storage requirements.

That should keep you at less than 60w I'd think. I wouldn't look at replacing the CPU unless you're very interested in reducing power consumption under load, but this is a costly option.

You could also spend some serious $$ on a more efficient power supply, something like 80+ platinum certified but again I don't think this would be worth your investment.
 
I disconnected the perc and all the harddisks, connected a 80gb Intel X-25 SSD and moved the Server2008 (which is my main vm). Lets see the results of that.

@Faugusztin:
No Perc but 4 X harddisk = 100W
No Perc but 2 X harddisk = 97W
Something is really wrong

@shnelson
The psu is a 80Plus (was recommended here on hardforum a few years back)
http://www.silverstonetek.com/product.php?pid=247
 
Last edited:
Indeed. 2 hard drives can't add 50W to the power consumption itself, unless the OS/hypervisor starts behaving completely different when they are present.

Try to check the BIOS settings if EIST and C-states are enabled.
 
You could also try measuring the power load without ESXI. I mean boot off a linux usbstick and do the same measurement.

2 hard drives can't add 50W to the power consumption itself, unless the OS/hypervisor starts behaving completely different when they are present.
Click to expand...
Hmm. Lots of CPU usage caused by drives being present?
 
Bios Settings:
Intel Hyper Threading = Enabled
Active Processor Cores = All
Turno Boost = Enabled
Power Technology = Energy Efficient - If i chance it to custom, then i can set EIST and Cstate+Pstate

Custom:
Eist = Enabled
Pstate = HW_ALL
CPU C3 = Disabled
CPU C6 = Enabled
Package C state limit = No limit

Should i set it to Power Efficient or custom?
 
I'm with them. That's too high. Something is wrong or your measuring device is wrong. I'd expect to see something like 40w idle there.
 
I updates the bios and checked the settings, seems ok. Power technology is set to Efficient at the moment.

It is now running with 1 ssd drive with esxi booted from usb and with 1 VM running. I'll let it run until tomorrow and check the Time/kWat.

Thanks for all the replies so far :)
 
I run the nearly the same setup (E3-1230. X9SCM-F, 32GB RAM, 1 USB drive, 2x dual NICS, 430W 80+ PS) and I idle at 39.2 watts and at peak hitting all cores is about 82 watts IIRC.
 
peanuthead said:
I run the nearly the same setup (E3-1230. X9SCM-F, 32GB RAM, 1 USB drive, 2x dual NICS, 430W 80+ PS) and I idle at 39.2 watts and at peak hitting all cores is about 82 watts IIRC.
Click to expand...

Seems like i should be able to get the same 40w in idle.

What storage solution do you use?
 
I agree you should be able to get near the same results. From the looks I would possibly retry flashing the BIOS. I use a Synology for part of my storage.
 
Just a small update :)
The system have now been running with 1 SSD disk for 17 hours and used 0,83kW, which average 49W pr hour.
Will connect 2 Sata drives to the motherboard and see how it works out.

Peanuthead@ i do actually have a synology DS412+ but always thought it was too slow for datastore.
 
LJensen said:
Just a small update :)
The system have now been running with 1 SSD disk for 17 hours and used 0,83kW, which average 49W pr hour.
Will connect 2 Sata drives to the motherboard and see how it works out.

Peanuthead@ i do actually have a synology DS412+ but always thought it was too slow for datastore.
Click to expand...

What specifically about it do you think is slow?
 
Read a few reviews where they said that it could be used, but it was not adviced since it was too slow/clunky. So i just assumed that they were right :)
How many vm's do you have running? Can the disk still spin down on the nas, if the vm's are running?
 
Sustained I run anywhere from 6-10 VMs.It all comes down to configuration. The disks could spin down but it would cause problems in the VM.
 
LJensen said:
Just a small update :)
The system have now been running with 1 SSD disk for 17 hours and used 0,83kW, which average 49W pr hour.
Will connect 2 Sata drives to the motherboard and see how it works out.
Click to expand...

New update :)

The system have now been running with 1 SSD and 2 sata harddisks for 8 hours and used 0,41kW, which average 51w pr hour.
The 2 sata harddisks have added 2w pr hour like they should.

Just connected 2 more drives to the motherboard, so lets see in 4-8 hours :)
 
New update :)

The system have now been running with 1 SSD and 4 sata harddisks for 18 hours and used 0,95kW, which average 53w pr hour.
The 2 extra sata harddisks have added 2w pr hour like they should, which was the same as the first 2.

Just connected the Perc 5, lets see if that is a power hungry fellow :)
 
Back on topic :)

With the Perc connected, 4 X sata harddisk and 1 ssd the Device, the device i use to check the power consumption now shows.
Hours : 5
kW: 0,41
0,82w pr hour (and i know that i can't say that, but don't know the right term. But i think most know what i mean)

Think i will just buy a bigger SSD drive and get some sort of daily backup for that.
 
If I may:

Runtime: 5 h
Energy consumption[1]: 0.41 kWh
Average power draw: 0.41 kWh / 5 h = 0.082 kW = 82 W

[1] Let's just ignore "consumption" since energy is never lost.
 
You must log in or register to reply here.
Back
Top