Server Specs for a Virtualization Server?

T

TechieSooner

Supreme [H]ardness
Joined
Nov 7, 2007
Messages
7,601
I'm looking at getting a nicely spec'd server to run several VMs on.

Looking at a 2TB disk space with 32GB of RAM on it.

My main question is if SCSI is worth going to on a VM like this. The price difference is almost double the entire cost, just by going to SCSI instead of SATA.

The apps are just propriety stuff. Nothing fancy like Exchange and BES which really needs some fast HDDs.
 
It really depends on your projected usage of the drives (how many users, what they will be used for, and what the actual read/write benchmark is). If they are just apps then you should be fine going SATA II. However, if you have high disk usage apps like a database then you might want to look into SAS.
 
32GB of ram will run you a TON of VMs. Our primary ESX nodes run with 8GB and each have 4-8VMs on them (with allocated memory anywhere from 512-3072) and we have room to grow. 32GB is a TON of memory for a single, isolated vm host. If you're doing anything moderately intensive on any of the vms, the disks will quickly become a bottleneck. This is especially true if you're planning on running raid 5 (5 disks with hotspare, this is what i'm assuming). The bottleneck will only become an issue even earlier with SATA drives.

Are you converting existing physical machines to vms? Gather performance metrics, try to get a feel for your IOP requirements. If you're sticking with 1 physical server and limited to 6 disks, SAS is going to give you about double the IOPS - especially on random writes, which your access pattern is going to be with all of those VMs on one volume. If I were trying to cram as many VMs onto one box with local storage as I could I'd go with SFF 15k SAS drives and raid10.
 
Especially with the transparent page sharing, if you are running a lot of very similar virtual machines (think all the same OS) you will save about 30% on memory. The typical consolidation 'target' is 3-5 virtual machines per CPU core ... ie a dual quad core = 24 - 40 virtual machines.

I see our 8 way boxes here at work with 24GB ram supporting 20'ish VMs with no problems but they keep moving around when DRS balances the cluster so I normally don't see that number unless I have a host offline for patching.
 
SCSI = Dead

you want fast you want SAS


As said, depends what you plan to run, do you want to run several Databases? or several other servers... the point of VM's is to use each part in te system to the best possible, if you want to run several databases then you need a dam fast I/O system.
 
Yea, I've been looking at going SAS but I really haven't wanted to go that way yet. I think it'd be more future proof than anything else though.

It'd be running SQL databases on each VM. One would be pretty intensive whereas the other is just a document retrieval system (only time SQL gets used is inputting a new document or running a query for an exiting one).
Other than that, I really wasn't planning on putting much else that's intensive. I was planning on using one VM for little apps (instant messaging client, door security system software, etc).

I may have to re-evaluate going SAS then. Pretty darn expensive though. Cheaper to upgrade in the future though (buy disks to stick into SAS, and not so many in the system iteself).
 
MrGuvernment said:
SCSI = Dead

you want fast you want SAS


As said, depends what you plan to run, do you want to run several Databases? or several other servers... the point of VM's is to use each part in te system to the best possible, if you want to run several databases then you need a dam fast I/O system.
Click to expand...

Serial Attached SCSI Oo
 
i know what SAS is..lol ;) i mean as in SCSI interface :) silly wabbit!


SAS isnt that expensive, you can buy a SAS controller and connect regular SATA harddrives to it...
i have a 3ware 9690SA with 7 normal Segate 500G ES2 SATA drives on it..
 
Yup you can connect regular SATA drives but just like your standard servers... It doesn't perform as fast as SCSI does.

I was looking at Dell's PowerVault SAS... Is that decent?
 
The last thing you want to do is run in to a performance issue or potentially bring down a large number of servers because you cheaped out on hard disks.

SAS all the way.
 
I'm leaning to SAS.

Question for you... Go RAID5 or RAID10?

RAID5 is obviously more cost effective, but for running virtual machines on them, is RAID10 worth it?
 
If you go with the powervault, for the most part you can mix and match drive sets. Meaning you can have five 15K drives and then 5 7.2K drives, and the enclosures don't have to be fully filled when you get em.

Depending on what kind of performance you'll need I Imagine a volume over a raid5 of the slower drives would be good enough for the low volume sql server, but you would probably want to go RAID10 for the high volume one, but the more drives you have spanned on a raid5/6 array the better your performance will be.
 
infamus said:
nvm i thought you said you did have BES or Exchange
Click to expand...
Yea, in another thread awhile back I did. That's sitting on a dedicated SCSI server.

Keiichi said:
If you go with the powervault, for the most part you can mix and match drive sets. Meaning you can have five 15K drives and then 5 7.2K drives, and the enclosures don't have to be fully filled when you get em.

Depending on what kind of performance you'll need I Imagine a volume over a raid5 of the slower drives would be good enough for the low volume sql server, but you would probably want to go RAID10 for the high volume one, but the more drives you have spanned on a raid5/6 array the better your performance will be.
Click to expand...

So, I'm thinking of getting 6 drives off the bat. So I'd have 4 used, 1 for parity, and 1 as a hot spare. As needs grow obviously I then get more usable.

Would you go that route or RAID10?
 
I would recommend RAID 10 if these servers are required to be highly available and fault tolerant. RAID 10 is faster than RAID 6 due to not having 2 parity bits and you can lose more drives with RAID 10.

Downside being you need to buy larger or more drives to make up for the loss in capacity, but the performance/reliability to cost ratio is something you always have to consider.

With multiple SQL database servers you will find RAID 10 being a better fit.
 
If you are starting out with 6 drives RAID 10 but I would suggest getting 7-8 so you have a hot/cold spare.
 
Pardon my lack of knowledge with RAID10, but it's basically setup
1 2 3 4 5 6 = Logical Volume 1
1 2 3 4 5 6 = Logical Volume 1

Mirrored+Striping. What if I lose both disk 1's???
In a RAID5 I know it'll rebuild itself but how does the RAID10 handle it?
 
Just FYI, there actually is a separate forum for virtualization topics: Virtualized Computing

Having said that; The SAS bandwagon on here is getting really really old.
All hail SAS!!!!111!!11!!eleven!!!

I call BS.
Most folks overestimate the IOPS they need by orders of magnitude.
There is no need to buy SAS if you can work with SATA just as well.

"Multiple SQL servers" don't automatically require SAS or Raid10. It all depends on what they are doing. Yes, if you run highly active transactional DBs, you need the I/O to go along with it, if your DB is just a department phone list that gets queries once a month, you don't need SAS.

Remember, bad advice is free, unless you take it, then it will cost you.

First, determine your need.
Second, size a solution to you need.

Future proof is bullshit. By the time future comes around the hardware cost of a total replacement with new technology will have dropped way below what you would spend today on a "future-proof" current technology system. If you want to, you can go with a SAS controller that you can hook SATA drives into, that way you can upgrade to SAS if you end up needing them.

Sticking 8 drives into a RAID10 array for virtual machines which is shared among all VMs is less optimal than creating dedicated arrays for machines that need them.

The amount of generalization applied to hardware recommendations is truly mind boggling to me. :(

Edit:
TechieSooner said:
Pardon my lack of knowledge with RAID10, but it's basically setup
1 2 3 4 5 6 = Logical Volume 1
1 2 3 4 5 6 = Logical Volume 1

Mirrored+Striping. What if I lose both disk 1's???
Click to expand...
Then you lose data. If you want to prevent that and stick to the basic Raid10 theme you will need Raid50.
 
TechieSooner said:
Pardon my lack of knowledge with RAID10, but it's basically setup
1 2 3 4 5 6 = Logical Volume 1
1 2 3 4 5 6 = Logical Volume 1

Mirrored+Striping. What if I lose both disk 1's???
In a RAID5 I know it'll rebuild itself but how does the RAID10 handle it?
Click to expand...

Then your array dies. Raid5 you lose any two disks and your array is dead. Yes, you can rebuild from a spare, but if you lose a second disk before the rebuild is done you're still dead.

When you lose a disk in raid10 and replace it, it is rebuilt from the mirrored disk. You could conceivably lose half the disks in your array (unlikely to only lose one from each mirror though) and still not lose data.
 
Thuleman said:
Just FYI, there actually is a separate forum for virtualization topics: Virtualized Computing

Having said that; The SAS bandwagon on here is getting really really old.
All hail SAS!!!!111!!11!!eleven!!!

I call BS.
Most folks overestimate the IOPS they need by orders of magnitude.
There is no need to buy SAS if you can work with SATA just as well.

"Multiple SQL servers" don't automatically require SAS or Raid10. It all depends on what they are doing. Yes, if you run highly active transactional DBs, you need the I/O to go along with it, if your DB is just a department phone list that gets queries once a month, you don't need SAS.

Remember, bad advice is free, unless you take it, then it will cost you.

First, determine your need.
Second, size a solution to you need.

Future proof is bullshit. By the time future comes around the hardware cost of a total replacement with new technology will have dropped way below what you would spend today on a "future-proof" current technology system. If you want to, you can go with a SAS controller that you can hook SATA drives into, that way you can upgrade to SAS if you end up needing them.

Sticking 8 drives into a RAID10 array for virtual machines which is shared among all VMs is less optimal than creating dedicated arrays for machines that need them.

The amount of generalization applied to hardware recommendations is truly mind boggling to me. :(

Edit:

Then you lose data. If you want to prevent that and stick to the basic Raid10 theme you will need Raid50.
Click to expand...



He already mentioned running a highly accessed SQL database along with other SQL databases. To be fair his definition of intensive may be different from yours, but you must account for the worst or you will regret it later.

How are you going to create multiple highly fault tolerant arrays for machines that need them with only 6 disks. You get 2 less fault tolerant arrays and if you are so sure that people are overestimating IOPS then why suggest a more "optimal" multiple array config.

Multiple arrays are a great idea if you have the amount of hard drives to do it. 6 is not that amount.
 
gimp0 said:
He already mentioned running a highly accessed SQL database along with other SQL databases. To be fair his definition of intensive may be different from yours, but you must account for the worst or you will regret it later.

How are you going to create multiple highly fault tolerant arrays for machines that need them with only 6 disks. You get 2 less fault tolerant arrays and if you are so sure that people are overestimating IOPS then why suggest a more "optimal" multiple array config.

Multiple arrays are a great idea if you have the amount of hard drives to do it. 6 is not that amount.
Click to expand...

Indeed. Independent arrays are nice, but you're not getting independent fault tolerant arrays for your VMs on a server with 6 disks and 32GB of ram (unless you have two ram hungry VMs).

If he's spec'ing out a server for 32GB of memory, he can run dozens of VMs. Even at moderate IO, that'll kill a handful of SATA drives in raid5 on random writes (which is what his usage pattern will regress to).
 
da sponge said:
Then your array dies. Raid5 you lose any two disks and your array is dead. Yes, you can rebuild from a spare, but if you lose a second disk before the rebuild is done you're still dead.
Click to expand...
In the future, I'd probably have more disks. 8 or more. In which case I can lose more than two.
In RAID10, the thing that worries me is if I lose two in particular, the whole array is dead. RAID5, I can lose any given 2 and it'll be fine (Assuming 8 or more later on).

gimp0 said:
He already mentioned running a highly accessed SQL database along with other SQL databases. To be fair his definition of intensive may be different from yours, but you must account for the worst or you will regret it later.
Click to expand...
I'm honestly not sure what intensity this propriety app will use. I don't think it'll be that bad. I think I could probably allocate a basic VM and it be fine.

At this point I'm kindof reminded about the "Future proof is BS" statement that he made. I tend to agree. By that point in time, there will be something bigger and better, plus costs will be less expensive. I'm thinking I should only be looking ahead a few months, at least from a hardware perspective.
 
gimp0 said:
.. but you must account for the worst or you will regret it later.
Click to expand...
See, that's a philosophy I just can't work with, especially when there's no well defined reference point from where to measure "worst".

From where I am looking the course of action is to measure and document current usage, then make an educated and reasonable estimate for short term (<6months) and mid-range (6-18months) growth, and then fit hardware to that need.

Blindly accounting for the worst leads to over provisioning and ultimately wasted money, and since money is tight, it makes good sense to not rush to judgment but to do this by the numbers..

gimp0 said:
How are you going to create multiple highly fault tolerant arrays for machines that need them with only 6 disks.
Click to expand...
I wouldn't.
My argument is more along the lines of determining whether a 6 disk setup is the right choice for the need/requirement. Perhaps part of the 6-disk limitation was the fact that the OP looked at SCSI or SAS disks. We also don't know yet which Hypervisor he has picked, which matters as well because ESX is a lot more strict when it comes to supported hardware than other products.
 
Thuleman said:
Blindly accounting for the worst leads to over provisioning and ultimately wasted money, and since money is tight, it makes good sense to not rush to judgment but to do this by the numbers..
Click to expand...
This is alot of it. I'm kindof lucky that money spent on IT at this point in the economy is even being considered...
It's also why I'm backpedaling on the SAS. I need to go easy(ier) on cost here, just what'll tide over until business picks back up.


Perhaps part of the 6-disk limitation was the fact that the OP looked at SCSI or SAS disks. We also don't know yet which Hypervisor he has picked, which matters as well because ESX is a lot more strict when it comes to supported hardware than other products.
Click to expand...

6 Disk limit is because 600GB SCSI disks are expensive as all hell. Plus 4 usable gives me enough space for what I need at this point.

Planning on just using Microsoft's HyperV. I can get decent deals on that through Dell.
 
Thuleman said:
See, that's a philosophy I just can't work with, especially when there's no well defined reference point from where to measure "worst".

From where I am looking the course of action is to measure and document current usage, then make an educated and reasonable estimate for short term (<6months) and mid-range (6-18months) growth, and then fit hardware to that need.

Blindly accounting for the worst leads to over provisioning and ultimately wasted money, and since money is tight, it makes good sense to not rush to judgment but to do this by the numbers..


I wouldn't.
My argument is more along the lines of determining whether a 6 disk setup is the right choice for the need/requirement. Perhaps part of the 6-disk limitation was the fact that the OP looked at SCSI or SAS disks. We also don't know yet which Hypervisor he has picked, which matters as well because ESX is a lot more strict when it comes to supported hardware than other products.
Click to expand...

"worst" was a poorly chosen word for what you elaborated on. I have seen alot of companies not account for growth and end up replacing hardware because of it. I look at multiple solutions and look at the cost/benefit between them to find the best value that meets my current and future needs.

While it is true techie left out some details on hardware I think he was just looking at the disk performance based on his situation and not wanting us to critique his entire implementation. If I were in his shoes I would post all details to cover any gotchas that people will point out.
 
:) I can't post what you'd be wanting (expected output/performance/usage) because I have no idea myself.
 
Or what Virtualization software would you recommend? Ideally I still want to run Windows as the host OS. This is so that I can install and mess with 1 backup utility rather than one for each VM.
 
TechieSooner said:
So, I'm thinking of getting 6 drives off the bat. So I'd have 4 used, 1 for parity, and 1 as a hot spare. As needs grow obviously I then get more usable.

Would you go that route or RAID10?
Click to expand...

Yep that would be the bare minimum I would recommend for any production environment. Until you get to having 10 disks, then I would go for either RAID5 or 6 with a hotspare.

IMO anything above 10 disks RAID10 becomes a bit of a liability and a unnecessary expense.

One of the things to help determine if you want to go either SAS or SATA is the size of the database and how many transactions it will be processing. I guess if you have 100GB+ databases that require thousands of transactions per second, then you probably could justify getting rather large 15K SAS drives.

Just keep in mind the more spindles (drives in the array) the better performance you'll have.

My virtualization server setup is a Dell PE2950 with WIn2k8 as a host and SQL server, web server, and a application server all running on seperate VMs. The nifty thing I did with my setup is that the Hyper-V server boots from iSCSI, which is a MD3000i. Which means no drives in the 2950.
 
Keiichi said:
One of the things to help determine if you want to go either SAS or SATA is the size of the database and how many transactions it will be processing. I guess if you have 100GB+ databases that require thousands of transactions per second, then you probably could justify getting rather large 15K SAS drives.
Click to expand...

Nothing like that. A dozen GB total, if that. I'm more worried about the access of the DB and not the creation.


What are you using for your Virtual Machine software? I'm thinking just going VMWare Server, it's free.
 
^ I use Hyper-V and manage via System Center VMM. Although you can get the free version of Hyper-V which can be managed by a windows machine remotely. Only thing is it's limited to 32GB of ram and limited to 4 CPUs.
 
Keiichi said:
^ I use Hyper-V and manage via System Center VMM. Although you can get the free version of Hyper-V which can be managed by a windows machine remotely. Only thing is it's limited to 32GB of ram and limited to 4 CPUs.
Click to expand...

Per VM, or total?

And is that 4 physical CPUs or 4 total logical?

How's the full-blown version work? It looks like Datacenter edition comes with it included unlimited VMs but that's pretty expensive to upgrade to that.
 
Any version of Server 2008 includes the Hyper-V role, though I don't know about any limitations on hardware or amount of machines. I'm running the standard version, I only need 4VMs, and I only have 32GB of RAM anyway.
 
This is off the top of my head, the free version and windows std (Hyper-v, not the entire os)is limited to using 32GB of ram. Enterprise and up don't have a limitation for the physical host, although guest OS is limited to 64GB of ram.

Depending on what your guest OS is it's limited to using up to 4 virtualized CPUs.

In terms of os licensing STD allows for one VM instance of the software, ENT allows for four, and Datacenter is unlimited. And of course you are only limited by your HW as to how many guests OS can run.

I think I made a mistake earlier about the 4CPU thing. the free and windows std editions are limited two two physical processors for the host, but it can present up to four virtualized CPUS to the guest OS.

I'm pretty sure the free version of ESXi has about the same limitations, To be honest 32GB of ram is the practical limit for most servers anyway.
 
You must log in or register to reply here.
Back
Top