• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Need new VM server, hardware questions

H

h0mez

Limp Gawd
Joined
Nov 6, 2006
Messages
444
This is what I am currently running, as you can tell I got 23 VMs on there, and I want to run another 20ish, and help reduce the load on this machine. But I am not sure the best route to go in putting together another box together as I want to build(obviously I purchased this one).

The way I look at it, I can get an Opteron, i5/i7 (I have an I7-970 I could use), or another Xeon. Not sure best bang for my buck would be, and limitations I would run into using a non-server cpu. What is more important? Cores vs MHz

Intel Xeon E3-1230 V2 looks to outperform my 2x E5520s with half the cores. at like 400 after cpu and mobo
http://www.newegg.com/Product/Product.aspx?Item=N82E16819117286

or 2 of Intel Xeon E5-2603 something like 800 for cpus and mb
http://www.newegg.com/Product/Product.aspx?Item=N82E16819117271



[MG] Uploaded with ImageShack.us[/IMG]
 
You have to see what you think you need performance wise on the additional VM's. Also, if you go AMD, then you can't leverage what you have now in a Cluster, no vMotion..etc. You CAN go with either of your choices and run a two node cluster with vMotion, etc.

To me it boils down to this, more cores you have, the more VM's you can run, higher MHz you have, more performance per VM, however if I recall the Scheduler diminishes each CPU core with each additional core and counts the 1'st core as full MHz cap, then it lowers each CPU count.

Of course, all this depends on your workload and applications.
 
Last edited:
Ok no AMD.

All those VMs do the EXACT same thing, so the performance use on that box is simply divided by 23.
 
Wouldnt it be easier to get the same make/model of the one you already have?
If the performance of the one you have now is good, then the choice is easy.
 
Wouldnt it be easier to get the same make/model of the one you already have?
If the performance of the one you have now is good, then the choice is easy.
Click to expand...

No, for the $ he can get a modern system. SKT1366 systems are still overpriced and expensive.
 
Is this for actual production or just testing/learning. If it's production, duplicate the box you have now. You know the performance, you know the quirks of the hardware, you know the price, it's a done deal. Don't play any games with it now, and you won't have to play any games with it at 3am someday. If it's testing, then do what you want. :)

Next question... If you are going to introduce a new host, are you planning on making them completely separate? Any thoughts on shared storage to enable higher functionality? Or just using the freeware license?
 
He should be using a paid license since he's using over 32GB of memory, anyway, that's my assumption.

Why would you consider skt1366 hardware when we have advanced two generations? You act like the new hardware would not work with his current setup, and that's not true.

Never in a million years would I tell a customer, just buy what you have because you know how it performs.
 
Nate7311 said:
Is this for actual production or just testing/learning. If it's production, duplicate the box you have now. You know the performance, you know the quirks of the hardware, you know the price, it's a done deal. Don't play any games with it now, and you won't have to play any games with it at 3am someday. If it's testing, then do what you want. :)

Next question... If you are going to introduce a new host, are you planning on making them completely separate? Any thoughts on shared storage to enable higher functionality? Or just using the freeware license?
Click to expand...

Its a test box, this is all personal stuff. I plan on doing a cluster(thanks for that Vader) when i get a second box. I dont want to buy the same thing, because I know I overpaid for it(being a dell) and didnt want to actually put together a server, but I want to now.

So basically I just want biggest bang for my buck, and I guess if im sticking with intel that would be a new ivy bridge. This being a test box and all do I want to get a xeon, or a regular ivy? I would save money on ram getting an i5 as opposed to a e3/e5. Or is the motherboard options on a server board a necessity?

This Intel Xeon E3-1230 V2 has the passmark score as my current 2x E5520s combined, but 4 cores vs 8, now way I would see equal performance, right?
 
Last edited:
Vader said:
He should be using a paid license since he's using over 32GB of memory, anyway, that's my assumption.

Why would you consider skt1366 hardware when we have advanced two generations? You act like the new hardware would not work with his current setup, and that's not true.

Never in a million years would I tell a customer, just buy what you have because you know how it performs.
Click to expand...

Hold on now. Until now, we didn't know if it was in a production environment or test. We didn't know much of anything other than rough demands and that he started with a Tier1 server and that he was looking to White-box a second. Would you reccomend that a customer white-box a production server? Regardless of technology.
 
White box whatever, dev, test whatever, it still bad advice. Have you seen the cost of 1366 server hardware? It's still crazy expensive compared to what the OP can buy within a single Ivy Bridge Xeon, etc and to answer your question, it really depends. They make some pretty solid barebones systems today from Supermicro..etc that work very well.

I would say if you are going to whitebox it, you better have spare parts laying around to rip and replace as to avoid downtime and each part should be warrantied. I'm not against whitebox per say, but I would also ask the customer how critical the applications that they are running are? Do they need a Gold type support..etc. I moreso favor a server from a solid vendor with a support package however.
 
Vader said:
I would say if you are going to whitebox it, you better have spare parts laying around to rip and replace as to avoid downtime and each part should be warrantied. I'm not against whitebox per say, but I would also ask the customer how critical the applications that they are running are? Do they need a Gold type support..etc. I moreso favor a server from a solid vendor with a support package however.
Click to expand...

Exactly. You were/are making a number of assumptions in your advice, and I'm not sure why.
 
The issue is about you telling the OP to buy the same hardware he has. Bottom line, bad advice. The cost of SKT1366 hardware is rediculous compared to what he can buy today...AND it still works fine together.

I've made my point, move on.
 
how do you figure? a second r610 is going to be less than a r620. you're also likely going to have to enable EVC if you have nehalem 1366 on one side and sandy bridge 2011 on the other side possibly losing some features from the sandy side.

i haven't looked into it much but a quick google on vmotion nehalem to sandy bridge pops back a lot of results. clearly there are at least minor issues to work around.
 
Vader said:
The issue is about you telling the OP to buy the same hardware he has. Bottom line, bad advice. The cost of SKT1366 hardware is rediculous compared to what he can buy today...AND it still works fine together.
Click to expand...

That's where we come into the assumptions. Nobody said it wouldn't, ever...

Re-read the post. With the information that we had to work with, which was little more than the initial server he was starting with and it's specs, and the # of VM's, there are too many unknows. We don't know the OPs comfort level and building/supporting a white box. We don't know his comfort level with VMWare. Why did he buy an OEM box to start with? Just because there are other opinions, doesn't make them wrong. With the advent of more info, it just makes them outdated.

We have more info to work with now, I'll let you focus on that. I'm out. OP, you have all the expertise you need with Vader.
 
Nate7311 said:
That's where we come into the assumptions. Nobody said it wouldn't, ever...

Re-read the post. With the information that we had to work with, which was little more than the initial server he was starting with and it's specs, and the # of VM's, there are too many unknows. We don't know the OPs comfort level and building/supporting a white box. We don't know his comfort level with VMWare. Why did he buy an OEM box to start with? Just because there are other opinions, doesn't make them wrong. With the advent of more info, it just makes them outdated.

We have more info to work with now, I'll let you focus on that. I'm out. OP, you have all the expertise you need with Vader.
Click to expand...

Exactly which is why I suggested the same unit since it performs well according to him.

But if im wrong im wrong! But do 2 wrongs make me right?! :confused:

Its all good.:cool:
 
Hagrid said:
Exactly which is why I suggested the same unit since it performs well according to him.

But if im wrong im wrong! But do 2 wrongs make me right?! :confused:

Its all good.:cool:
Click to expand...

Don't fight over me guys :)

I bought the dell in the first place because I wanted to finance it, and it was my first experience into the server world and didnt want to deal with hardware incompatabilites, just wanted something that worked out of the box. Now, basically I have maxed that box out, and just need moar power. Thats why I came to search you guys out, wasn't sure the route to go. I'll probably end buying an Ivy bridge something with 4 cores and HT, then learn and figure out how to setup the cluster since that seems like the best way to reduce some of the load of my already maxed box.

Thanks for all the help.
 
no you arent, unless its a dev preview chip.

*edit*

oh, wait the single socket dual/quad core rebadged desktop parts. forgot about those. sorry i meant real server heardware, nobody is shipping anything newer than sandy-e.
 
Last edited:
Buy a new Xeon Ivy Bridge, that is what I am doing right now to upgrade my old server. Most bang for the buck now days IMHO.
 
I believe the 1155 chipset only supports a max of 32 gig memory so if you go with the 1230 you are going to have less memory than your current setup.
 
Yea that should be fine, I'm going to grab an E3-1230 V2, any ideas on what mobo I should get? I've been looking at various supermicro boards, too many to choose from. And one random question I see lots of people install esxi on a usb pen drive what's the point of doing that?
 
h0mez said:
Yea that should be fine, I'm going to grab an E3-1230 V2, any ideas on what mobo I should get? I've been looking at various supermicro boards, too many to choose from. And one random question I see lots of people install esxi on a usb pen drive what's the point of doing that?
Click to expand...

Just so you don't waste a hard drive on the ESXi installation. The hypervisor gets loaded into memory anyways so it really doesn't matter too much what media you're booting from.
 
madrebel said:
how do you figure? a second r610 is going to be less than a r620.
Click to expand...

This isn't necessarily true. I configure and price IBM and Dell systems out quite regularly and the new E5 R620/R720 and 3550M4/3650M4 actually cost LESS in many cases than the R610/R710 and 3550M3/3650M3 predecessors. In additional, you really need to take into account performance. The E5 processor significantly and vastly outperforms the E55xx and E56xx processors and has more cores to begin with than the E55xx.


Just so you don't waste a hard drive on the ESXi installation. The hypervisor gets loaded into memory anyways so it really doesn't matter too much what media you're booting from.
Click to expand...


This. In all my enterprise accounts, we hardly ever use them for VMware anymore. there's typically a USB memory key or Flash SD card that we boot ESXi from. They usually vary in size from 2GB to 8GB.
 
Not to contradict, but my pricing/configuration experience w/ Dell has been a bit different. Until you come close to maxing out an R610, it still has the price advantage. Obviously it loses the technical advantage, due to updated proc support, Max RAM, dual SD, etc.
 
if you have a good dell rep you can get a few more points of the 610 as they're willing to move them out of inventory as fast as possible too.

no question the x20s are better, i just ordered 40 of them, they aren't cheap though.
 
Any ideas on a mobo? So many to choose from and various people having issues with the Ivy and bios versions, and then trying to run 4x8gb on them..
 
What are you using for storage? Do you need a lot of network bandwidth? Any other add-in cards?


It seems like one IVB xeon and 32GB of memory might be a bit tight for that workload, especially if you're hoping to offload a couple of VMs from your current box. The only issue is that if you're looking for bang for the buck, you might be just as far ahead building 2x of those versus buying a 6 core SNB-E.
 
That box has 3 1TB spinpoints setup for raid 5, i might add another and go raid 10. I don't need alot of network bandwidth(yet), I will i'm assuming if I try to cluster. So that means getting vlan capable switch etc. These VMs only each need like 12gb of space each so at ~23 VMs thats only 280ish gbs. So it might make sense to raid some SSD drives in the future.

I figure (tell me if im wrong), that with this 1230 V2 I can offload a few of those VMs, like 10 at least, and get another box later? I don't really have a desire to build a identical 2xE55XX box, there's better options for performance/price. I would like a box that has identical cpu power, but not sure what the cheapest way to do that is, 32gb of ram would probably be fine in a 2nd box...

I found this board, and seems like a no brainer for Ivy bridge and NIC compatibility.

http://ark.intel.com/products/67494/Intel-Server-Board-S1200BTLR?wapkw=s1200btlr
 
You shouldn't have an issue running that same workload on the 1230 V2 but if I understood your original post you wanted to put 20 more VMs up + take 10 of those off your current box. So like 30VMs in total.

You definitely want to look into getting a switch that is managed and LACP capable. It would allow you to trunk 2 ethernet ports as 1 for a 2gbps connection. You may or may not need it, but without knowing anything about the VMs we don't really know if it's an issue. Right now they might be moving traffic between VMs, but if it's on the vswitch then you have a lot more bandwidth available. (1gbps per vm vs 1gbps for all VMs)

Since you're running VMware you definitely will need a hardware RAID card because it doesn't support software raid. (Guessing you're using a PERC card for RAID 5 right now) I would think that trying to use the drives in JBOD configuration on the built in controller wouldn't be able to handle the I/O. (Unless there is very little disk activity on each VM) I'm not exactly sure what type of RAID is onboard the board you linked but I'm thinking it's still software based.

Just bringing up a few points as it sounds like you'll for sure need at least 1 PCIe 2.0 x8 slot and dual GBe NICs.
 
bman212121 said:
You shouldn't have an issue running that same workload on the 1230 V2 but if I understood your original post you wanted to put 20 more VMs up + take 10 of those off your current box. So like 30VMs in total.

You definitely want to look into getting a switch that is managed and LACP capable. It would allow you to trunk 2 ethernet ports as 1 for a 2gbps connection. You may or may not need it, but without knowing anything about the VMs we don't really know if it's an issue. Right now they might be moving traffic between VMs, but if it's on the vswitch then you have a lot more bandwidth available. (1gbps per vm vs 1gbps for all VMs)

Since you're running VMware you definitely will need a hardware RAID card because it doesn't support software raid. (Guessing you're using a PERC card for RAID 5 right now) I would think that trying to use the drives in JBOD configuration on the built in controller wouldn't be able to handle the I/O. (Unless there is very little disk activity on each VM) I'm not exactly sure what type of RAID is onboard the board you linked but I'm thinking it's still software based.

Just bringing up a few points as it sounds like you'll for sure need at least 1 PCIe 2.0 x8 slot and dual GBe NICs.
Click to expand...

Thanks, I'll change up the switch I was going to grab. I for some reason didn't realize I would need shared storage to setup a cluster. I'll have to set something up for that and then that will utilize the additional NICS.
 
I feel very dumb for asking all these questions. But after talking with an IT admin buddy and him looking at my current setup(he couldn't believe those VMs were using all that CPU). Well it was the case, my datasore IO is very low 7-10 on avg, he informed theres really no reason to cluster. And that really these VMs are not important in terms of redunancy/backup etc, don't require server grade hardware etc. Well I already bought the CPU E3-1230 V2, and being able to save money on the mobo and ram by getting desktop hardware will allow me to probably build another server.

So question is(and im sorry again for being a noob), is there anything specific I should look for in a regular desktop mobo? GIGABYTE GA-Z77-D3H or the GA-B75M-D3H, obviously the NICs might not be compatible(I can grab dual port Intel of ebay), but they should all be able to do virtualization with no problems right?
 
As long as that processor works in the board you decide to get it shouldn't really be an issue. Most boards now support VT-x so as long as you enable it everything should work well. I've run ESXi on a Q6600 with a 780i board and everything is detected and worked fine.

You can also check out: http://www.vm-help.com//esx40i/esx40_whitebox_HCL.php

http://ultimatewhitebox.com/

You might be able to pick a board from the list of hardware that works with your CPU and someone has already tested and figured out what works and what doesn't.
 
You must log in or register to reply here.
Back
Top