Small Datacenter

[Wrench00]

So some customers have expressed the want to get rid of their servers. They don't want to go to the public cloud (Azure, Amazon) due to the fact they don't want to have their information cross US/Can border, also a lot of them can't. Plus its a good way for me to ensure my own employment.

Any who so I have 3 hosts and a SAN and other networking equipment. I am a VMWare guy, but I also do hyperv. I am currently debating where I should go with vmware or HyperV. The cost differences are fairly small for me but Vmware is slightly more, I am a VPP and MPN Silver partner so I have access to everything. I have recently started playing with SCCM and SCVMM, with these tools I did find that MS products are much more labour intensive to setup.

I am a small business owner and I am self employed. I am looking at a fairly big investment in the datacenter costs.

Any recommendations from guys that run a datacenter.

 

[dandragonrage]

Unless you're a Windows-only shop and don't need IOMMU, avoid the s*** out of Hyper-V in favor of VMWare or Xen. Personally I find it a bit insulting that Microsoft felt it had to enter the virtualization market (which was already fine) with such a limited product. But of course, Microsoft will try to take over every single PC-related market, without thinking it needs to actually release a quality product. Microsoft's strategy is "We're MICROSOFT, people will buy our product regardless of how crappy it is." And people do! So frustrating! I recommend using Hyper-V for virtualization about as much as I recommend using IE for web browsing: not at all.

Can't beat VMWare's management tools, either.

 

[Benzino]

So are you looking to build your own data center to host your customer's VM's? Do you have the facilities for it? Its dreamy to build your own DC and host clients VM's until you look into SLA's.

 

[renixinq]

First thing is first, can you tell us a little bit about the facilities? Are you planning on running this out of your home? Are you getting some rack space at a company? Because while you can debate HyperV vs VMware all day long I think the bigger question is around how you are going to provide this solution to your customers.

 

[Thuleman]

Because while you can debate HyperV vs VMware all day long I think the bigger question is around how you are going to provide this solution to your customers.

Exactly, this. Reselling becomes a matter of how it's metered, how it's charged.

TBH, I wouldn't in a million years set up hosting someone's VMs out of my basement. The liability alone would make me cry myself to sleep every night.

I would find a local/regional company that runs datacenters and just resell their product with your customer support as value added service. You get the hardware/network SLA from the datacenter people who are good at what they do and then you add your own service in managing those VMs/servers for your customers. Done and done. Sleep easy.

 

[dave99]

2nd or 3rd the bad of idea of running this at home if that is the plan. Rent a rack in a solid datacenter. You'll need the internet bandwidth anyway unless you have fiber at home.

 

[Wrench00]

I have a Tier 1 Datacenter disposable to me. It was used by a Financial institutions, its been vacated now so I pretty well much have access to I think 10-12 racks just to me.

 

[Benzino]

I have a Tier 1 Datacenter disposable to me. It was used by a Financial institutions, its been vacated now so I pretty well much have access to I think 10-12 racks just to me.

That's handy. I imagine that it is still actively managed then? If so, you might have nailed a gold mine.

 

[Thuleman]

Unless you're a Windows-only shop and don't need IOMMU, avoid the s*** out of Hyper-V in favor of VMWare or Xen. Personally I find it a bit insulting that Microsoft felt it had to enter the virtualization market (which was already fine) with such a limited product. But of course, Microsoft will try to take over every single PC-related market, without thinking it needs to actually release a quality product. Microsoft's strategy is "We're MICROSOFT, people will buy our product regardless of how crappy it is." And people do! So frustrating! I recommend using Hyper-V for virtualization about as much as I recommend using IE for web browsing: not at all.

It's pretty clear that you are not a MS fan, but speaking as a professional I take issues with people who evangelize one thing or the other. It's my responsibility to find, implement, and maintain, the best possible solution for the entity that pays me to do just that. In order to do that I feel that I cannot blindly dismiss any vendor or any technology. If I would focus on a single vendor all the time I would fail the people who hired me because they don't pay me to implement what I like but to implement what's good for the entity. Ideally those two are the same, but they don't have to be.

You go on and on and on about how the demand on the storage network depends and 1GB can be insufficient storage network bandwidth for some virtualization cases (even though 1GB storage links are still very much the norm in a lot of datacenters and there are many cases where that link never gets even close to saturation), the same is true for virtualization technologies.

Microsoft was very late to virtualization, but their products have gotten better over the years and MS has been gaining market share because for some deployments using the MS-only stack just makes good sense.

MS is steadily gaining market share, in part because of attractive licensing costs and in part because their technology is actually getting better and integrates better with the data center suite of MS products.

None of us knows what will happen in 12, 24, 48 months when it comes to virtualization technology, but limiting yourself with a zeal to a single technology/vendor isn't good advice no matter how you look at it.

 

[Child of Wonder]

It's pretty clear that you are not a MS fan, but speaking as a professional I take issues with people who evangelize one thing or the other. It's my responsibility to find, implement, and maintain, the best possible solution for the entity that pays me to do just that. In order to do that I feel that I cannot blindly dismiss any vendor or any technology. If I would focus on a single vendor all the time I would fail the people who hired me because they don't pay me to implement what I like but to implement what's good for the entity. Ideally those two are the same, but they don't have to be.

You go on and on and on about how the demand on the storage network depends and 1GB can be insufficient storage network bandwidth for some virtualization cases (even though 1GB storage links are still very much the norm in a lot of datacenters and there are many cases where that link never gets even close to saturation), the same is true for virtualization technologies.

Microsoft was very late to virtualization, but their products have gotten better over the years and MS has been gaining market share because for some deployments using the MS-only stack just makes good sense.

MS is steadily gaining market share, in part because of attractive licensing costs and in part because their technology is actually getting better and integrates better with the data center suite of MS products.

None of us knows what will happen in 12, 24, 48 months when it comes to virtualization technology, but limiting yourself with a zeal to a single technology/vendor isn't good advice no matter how you look at it.

This.

Hyper-V is not a shit product anymore like the 2008 and 2008 R2 versions were. Microsoft has done a great job catching up to VMware with features in Hyper-V 2012 and have plans to catch up even more while innovating new features in Hyper-V 2012 R2 due later this year. That coupled with their licensing costs for existing MS shops makes them a hypervisor that can't be ignored.

A few weeks ago I gave a presentation to 20 people at my company regarding the important differences between Hyper-V and VMware not covered in the FUD and marketing hyper seen online. None of them knew a thing about Hyper-V but did know a great deal about VMware. They were surprised to see how much Hyper-V had closed the gap with the latest release. Being a VMware veteran of 7 years, a dual VCAP, and a consultant whom my clients trust, I've found it important to familiarize myself with Hyper-V and the System Center management suite because my customers are asking about it. I can either turn down their business and do everything I can to make them stay with VMware or I can adapt to best guide them in choosing, designing, and implementing the hypervisor and management suite that fits their needs.

 

[-Dragon-]

Personally I find it a bit insulting that Microsoft felt it had to enter the virtualization market (which was already fine) with such a limited product.

LOL? Already fine? Did you already forget the vRAM tax and how people had to deploy lots of tiny VM hosts instead of single large ones with lots of CPUs and RAM to avoid the heavy penalties VMW placed on doing virtualization properly? Do you forget how as soon as MS launched Hyper-V v3 that the next ESX update did away with that monopolistic bullshit and went back to the more reasonable pricing model?

 

[Wrench00]

Good lord I started vmware vs hyperv war. *sigh*.

Not very constructive.

 

[Hagrid]

This.

Hyper-V is not a shit product anymore like the 2008 and 2008 R2 versions were. Microsoft has done a great job catching up to VMware with features in Hyper-V 2012 and have plans to catch up even more while innovating new features in Hyper-V 2012 R2 due later this year. That coupled with their licensing costs for existing MS shops makes them a hypervisor that can't be ignored.

A few weeks ago I gave a presentation to 20 people at my company regarding the important differences between Hyper-V and VMware not covered in the FUD and marketing hyper seen online. None of them knew a thing about Hyper-V but did know a great deal about VMware. They were surprised to see how much Hyper-V had closed the gap with the latest release. Being a VMware veteran of 7 years, a dual VCAP, and a consultant whom my clients trust, I've found it important to familiarize myself with Hyper-V and the System Center management suite because my customers are asking about it. I can either turn down their business and do everything I can to make them stay with VMware or I can adapt to best guide them in choosing, designing, and implementing the hypervisor and management suite that fits their needs.

I guess I should go back and try the new hyper-v. I tried the older version and hated it.

Though I just do this stuff for fun, since I don't work in the industry.

 

[Grentz]

I would definitely focus on your facilities (Power, Network, Climate Control), those can be real killers to making these sort of solutions work. It seems easy at first, but you need to also factor in maintenance, repair costs, uptime SLAs, etc. For most smaller operations (a few racks), a managed facility always works out better from a long term ROI. Especially the Power and Climate control can add up to big cost over the long term for a sizable operation.

You mentioned you had access to a facility, but just wanted to clarify the maintenance/long term aspect of the costs too. First time you have to redo the fire suppression to get up to code or replace an HVAC rig, that can really hit your bottom line hard.

I would look very hard at the backup/SLA side of things as well. Telcos, as I am sure you know, can be a real bitch to work with and often have issues that you cannot fix (always aggravating waiting for them to bring things back up when you have clients bitching at you). Also, you don't want to be on the hook for losing someone's data as you can get in hot water very quick.

Overall though, if you have the know how, facilities, and client interest, I think it is an awesome idea. I would much rather colo for my clients than set them up with something like Amazon. Cloud has its place, but I have not seen it that effective for at least my mix of client apps/needs beyond basic collaboration (Email/IM/Voice).

BTW, the US/CAN issue is a non issue with Amazon at least. You can get them to dedicated your info to a specific site. In fact, the enterprise contracts I have worked with always specify a master site and DR site.

 

[Grentz]

I guess I should go back and try the new hyper-v. I tried the older version and hated it.

Though I just do this stuff for fun, since I don't work in the industry.

I hated the old Hyper-V, but love Hyper-V 2012. Its the best Hypervisor for an SMB Windows Shop IMO.

Unless I was running a very large data center, I would go with Xenserver and Hyper-V managed with SCVMM.

 

[Wrench00]

I would definitely focus on your facilities (Power, Network, Climate Control), those can be real killers to making these sort of solutions work. It seems easy at first, but you need to also factor in maintenance, repair costs, uptime SLAs, etc. For most smaller operations (a few racks), a managed facility always works out better from a long term ROI. Especially the Power and Climate control can add up to big cost over the long term for a sizable operation.

You mentioned you had access to a facility, but just wanted to clarify the maintenance/long term aspect of the costs too. First time you have to redo the fire suppression to get up to code or replace an HVAC rig, that can really hit your bottom line hard.

I would look very hard at the backup/SLA side of things as well. Telcos, as I am sure you know, can be a real bitch to work with and often have issues that you cannot fix (always aggravating waiting for them to bring things back up when you have clients bitching at you). Also, you don't want to be on the hook for losing someone's data as you can get in hot water very quick.

Overall though, if you have the know how, facilities, and client interest, I think it is an awesome idea. I would much rather colo for my clients than set them up with something like Amazon. Cloud has its place, but I have not seen it that effective for at least my mix of client apps/needs beyond basic collaboration (Email/IM/Voice).

BTW, the US/CAN issue is a non issue with Amazon at least. You can get them to dedicated your info to a specific site. In fact, the enterprise contracts I have worked with always specify a master site and DR site.

Sorry I just realized I made a typo its a tier 4 datacenter. The facilities are not an issue. My question is has any one setup a larger scale Multitenant Cluster on either vcenter or hyperv if so what are your recommendations?

There are thing that do annoy me about HyperV, non scsi boot drive and, pxe network cards. In this respect vmware is much better.

 

[Grentz]

Sorry I just realized I made a typo its a tier 4 datacenter. The facilities are not an issue. My question is has any one setup a larger scale Multitenant Cluster on either vcenter or hyperv if so what are your recommendations?

There are thing that do annoy me about HyperV, non scsi boot drive and, pxe network cards. In this respect vmware is much better.

2012 R2 will at least fix the SCSI boot drive

I did a lot of research around multitenant architecture for some security concerns, and quite frankly the most difficult part is the network. While easy at first, it is very easy for machines to hop networks due to misconfiguration (which is easy to do accidentally) and ensuring that nothing rogue gets out there. In that regard, I think VMWare is still better, plus the have products aimed at multi-tenant deployments.

I believe 2012 R2 for hyper-v has a few things coming that will help as well, though not 100% sure there.

Overall I would be thinking more along the lines of what are the risks in the apps your clients are hosting. If it is all pretty basic stuff, its a no brainer. If they are doing more intensive/secure/critical things, it takes more upkeep and ongoing thought. You would likely not be able to share as much of the infrastructure and deployments for example.

 

[Wrench00]

2012 R2 will at least fix the SCSI boot drive

I did a lot of research around multitenant architecture for some security concerns, and quite frankly the most difficult part is the network. While easy at first, it is very easy for machines to hop networks due to misconfiguration (which is easy to do accidentally) and ensuring that nothing rogue gets out there. In that regard, I think VMWare is still better, plus the have products aimed at multi-tenant deployments.

I believe 2012 R2 for hyper-v has a few things coming that will help as well, though not 100% sure there.

Overall I would be thinking more along the lines of what are the risks in the apps your clients are hosting. If it is all pretty basic stuff, its a no brainer. If they are doing more intensive/secure/critical things, it takes more upkeep and ongoing thought. You would likely not be able to share as much of the infrastructure and deployments for example.

Its mostly Windows stuff some Linux deployments. Nothing super critical.

FYI

I work with Governments and publicly funded institutions which pretty well much kills any idea of going to a major public cloud such as amazon or azure. I've spoken with both and they couldn't guarantee that the data will not cross the border. The whole NSA thing is scaring the shit out of lot of people.

 

[Grentz]

Its mostly Windows stuff some Linux deployments. Nothing super critical.

FYI

I work with Governments and publicly funded institutions which pretty well much kills any idea of going to a major public cloud such as amazon or azure. I've spoken with both and they couldn't guarantee that the data will not cross the border. The whole NSA thing is scaring the shit out of lot of people.

GovCloud is the specific area of Amazon Cloud that will guarantee it. You will pay a bit more and there are some entry requirements (have to justify the need), but it is guaranteed. Probably more of a non-selling point though for what you want to do, so just one to stay aware of.

http://aws.amazon.com/govcloud-us/

I have used them for deployments that are way more sensitive than just simplying being a local government or public institution.

 

[Child of Wonder]

Sorry I just realized I made a typo its a tier 4 datacenter. The facilities are not an issue. My question is has any one setup a larger scale Multitenant Cluster on either vcenter or hyperv if so what are your recommendations?

There are thing that do annoy me about HyperV, non scsi boot drive and, pxe network cards. In this respect vmware is much better.

SCSI boot, synthetic vNIC PXE boot, live VHDX resize, live cloning, and a slew of other features are due out with Hyper-V 2012 R2 later this year.

It's a great time to be a virtualization guru!

 

[Wrench00]

SCSI boot, synthetic vNIC PXE boot, live VHDX resize, live cloning, and a slew of other features are due out with Hyper-V 2012 R2 later this year.

It's a great time to be a virtualization guru!

Looks pretty good.

How do you handle switches with in HyperV ? I find them very confusing as hell compared to Vcenter.

 

[Grentz]

SCSI boot, synthetic vNIC PXE boot, live VHDX resize, live cloning, and a slew of other features are due out with Hyper-V 2012 R2 later this year.

It's a great time to be a virtualization guru!

mmmm, live cloning,

 

[Grentz]

Looks pretty good.

How do you handle switches with in HyperV ? I find them very confusing as hell compared to Vcenter.

They are not too bad via SCVMM, but SCVMM adds a lot of overhead on small deployments (since it has pretty large requirements for the VM that is running it and SQL, etc.).

I dont mind the network setup via the Hyper-V manager though it does not have as graphical of a view as vCenter.

 

[Child of Wonder]

Looks pretty good.

How do you handle switches with in HyperV ? I find them very confusing as hell compared to Vcenter.

One virtual switch per NIC or NIC team. VNICs created off the vswitch.

On my lab hosts I have three vswitches:

One connected to a NIC team. 3 vNICs presented to Management OS and VLANed: management, live migration, and cluster heartbeats. All VM vNICs run on this vswitch.

Two vswitches for SMB each with a vnic presented to the management OS.

Think of hyperv as you would legacy ESX but without port groups. Everything should be a vNIC - both the NICs presented to the Management OS and VMs. What does suck is needing to specify VLANs at the vNIC level rather than simply assigning to a port group. Power she'll comes in handy there especially if you don't use SCVMM.

 

[renixinq]

It sounds like your more comfortable in VMware (which you mentioned) so I'm wondering if the price difference is worth the time you'll invest in getting your HyperV expertise up to par with your VMware knowledge. And is it worth you possibly mis-configuring something in HyperV that might lose you a customer vs going with what you know? I'd be willing to bet any issues you have with HyperV will have some sort of work around to address the issue. I'm a VMware guy so I'll stick with VMware not because I don't think HyperV can't cut it but because I don't want to reinvest in a different technology.

That said if the price difference is significant enough I'd wager HyperV with a multi-tenant setup will work just as well as a VMware setup. It's just learning the nuances that go along with it.

 

[MichaelCorsa]

FYI. Cisco announced yesterday the availability of its Nexus 1000V virtual appliance for the Hyper-V platform. I bet things are getting hotter now between VMware and Microsoft.

http://blogs.technet.com/b/schadinio/archive/2013/06/11/cisco-nexus-1000v-available-for-hyper-v.aspx

I would like to mention that I work with both of these products and especially the last 1.5 years the only difference between them is ...... almost nothing. In most environments that I have come across, 2 to 5 hosts in a cluster plus a storage and 1 to 5 other physical servers(that's almost 80-90% of what every one us will encounter in our daily work, I believe), there was no difference in speed, practicality and ease of management. The only thing that I can say is that VMware is a bit more expensive for the same solution than Microsoft, and is not as easy to deploy along with other hypervisors (vCenter cannot manage other hypervisors but SCVMM2012 has enough functionality to support other than HyperV).

Also check the upcoming Windows 2012 R2 release and the features that it will have in the virtualization field. http://windowsitpro.com/hyper-v/top-10-windows-server-2012-r2-hyper-v-new-features. This is a year that I wouldn't like to be an employee in VMware (...yeah sure, any position is kindly accepted ).

 

[Deleted member 16178]

So some customers have expressed the want to get rid of their servers. They don't want to go to the public cloud (Azure, Amazon) due to the fact they don't want to have their information cross US/Can border, also a lot of them can't. Plus its a good way for me to ensure my own employment.

Any who so I have 3 hosts and a SAN and other networking equipment. I am a VMWare guy, but I also do hyperv. I am currently debating where I should go with vmware or HyperV. The cost differences are fairly small for me but Vmware is slightly more, I am a VPP and MPN Silver partner so I have access to everything. I have recently started playing with SCCM and SCVMM, with these tools I did find that MS products are much more labour intensive to setup.

I am a small business owner and I am self employed. I am looking at a fairly big investment in the datacenter costs.

Any recommendations from guys that run a datacenter.

It's going to be unreasonably expensive to start your own datacenter. Start off by renting a colocation space at a third-party facility. That will give you the control that you want, allow you to charge a tidy mark-up for your clients, and prevent you from having to deal with a majority of the setup, overhead, and ongoing costs of a proper datacenter.

 

[Wrench00]

It's going to be unreasonably expensive to start your own datacenter. Start off by renting a colocation space at a third-party facility. That will give you the control that you want, allow you to charge a tidy mark-up for your clients, and prevent you from having to deal with a majority of the setup, overhead, and ongoing costs of a proper datacenter.

I have no problem with having the hardware and costs.

Again I was just looking at advice. I think I am just gone stick with vmware for now. Its a small implementation and I am not going to be running 1000s of vms.

 

[Deleted member 16178]

I have no problem with having the hardware and costs.

Again I was just looking at advice. I think I am just gone stick with vmware for now. Its a small implementation and I am not going to be running 1000s of vms.

If you don't mind me asking, what is your budget goal for this datacenter?

 

[Wrench00]

If you don't mind me asking, what is your budget goal for this datacenter?

I don't talk about finances on the internet.

My Goal is to run about 60 VM's, I currently have the hardware capacity of doing so without an issue. The question I wanted to ask is should I go with VMware or HyperV, it will be almost all MS shop.

 

[Deleted member 16178]

I don't talk about finances on the internet.

My Goal is to run about 60 VM's, I currently have the hardware capacity of doing so without an issue. The question I wanted to ask is should I go with VMware or HyperV, it will be almost all MS shop.

Looking forward to seeing this datacenter that houses 60 VM's.

What you need is 1-2 servers and (perhaps) a NAS device in a colocation facility. We are talking probably around 4U total. Should end up costing around $200-$300 per month. I don't think "datacenter" was the right term.

Regarding VMware vs. Microsoft, what is your current licensing posture for MS? If you don't need clustering or HA, you could go with the free version of ESXi. If you have to buy VMware, you are looking at a couple grand for 2 licenses + vCenter. If you are on a budget, I would also suggest considering Proxmox. Unfortunately, I am not up to date on Hyper-V costs.

 

[Wrench00]

Looking forward to seeing this datacenter that houses 60 VM's.

What you need is 1-2 servers and (perhaps) a NAS device in a colocation facility. We are talking probably around 4U total. Should end up costing around $200-$300 per month. I don't think "datacenter" was the right term.

Regarding VMware vs. Microsoft, what is your current licensing posture for MS? If you don't need clustering or HA, you could go with the free version of ESXi. If you have to buy VMware, you are looking at a couple grand for 2 licenses + vCenter. If you are on a budget, I would also suggest considering Proxmox. Unfortunately, I am not up to date on Hyper-V costs.

I am sticking with VMWare. I spoke with vmware and I can do a cross specialization as a VSP (already VPP). As a VSP I get IUL which are like 50%-70% off, and I can even do a SPLA with it so per month its very little money. I also sat down and did some math with MS stuff I ended up coming up with very similar number. I just recently built a HyperV cluster with SCCM and there is a Huge amount of overhead and labor involved.

I will be hosting other people servers I will not be providing their own or they can SPLA.

I have a tier 4 datacenter (tier 3 really) I get 24U for about 200 Hundred dollars per month for 6 months.

I have 3 Hosts (2U) 6U+ 3U San. 2 Switches 1U and 1 U router. The deal is that I will take over an entire 48 U for 800 dollars a month in 6 months.

 

[Thuleman]

As a VSP I get IUL which are like 50%-70% off

Sounds like the person you were talking to at VMware forgot to tell you that:

"Internal use licenses are for the Partner‘s internal IT needs only and are not for hosting, resale or distribution to end customers or to any third party."

 

[Wrench00]

Sounds like the person you were talking to at VMware forgot to tell you that:

"Internal use licenses are for the Partner‘s internal IT needs only and are not for hosting, resale or distribution to end customers or to any third party."

I know this IUL is for mine but I also get VSP licensing which is also heavily discounted.