Considering Migrating from ESXi to KVM, Suggestions?

[Zarathustra[H]]

Hey all,

I am getting really tired of dealing with the commercial behemoth that is VMWare. When I originally started my Installing ESXi 5.0 on my server in my basement 4 years ago, the alternatives didn't seem quite ready, but now - by all accounts - they are.

I know I'm not a paid subscriber, and thus not their priority, but it still annoys me that I don't get the latest bug fixes in Update 1a and 1b, for instance.

I have a standalone ESXi 6 hypervisor running less as a lab, and more as a "home production system" in my basement, hosting my NAS, my TV DVR, my Router/Firewall, etc. etc.

I have 10 guests in total.

First of all, what frontend do you guys recommend? I hear good things about ProxMox. Are there any others I should look into before deciding?

Are there any tips to converting disk images over to a format that KVM can utilize, or do I have to reinstall, and re-set up all my guests?

Are there any limitations to KVM I should be aware of, that I might not realize it doesn't do, since I am used to ESXi?

Features I use are PCI passthrough, blockdevice passthrough (if I just want to pass one drive through, without passing the entire controller), vnic's vswitches, etc.

I'd appreciate any input you guys might have!

--Matt

 

[PigLover]

Proxmox is by far the most complete and easiest to use KVM front end (its actually more than that...). But you might find dealing with the attitude from the "team" at Proxmox just as frustrating as dealing with VMWare.

 

[Zarathustra[H]]

Proxmox is by far the most complete and easiest to use KVM front end (its actually more than that...). But you might find dealing with the attitude from the "team" at Proxmox just as frustrating as dealing with VMWare.

Do they have a history of blacklisting drivers for hardware to spite whitebox users?

Seriously, I feel like every step of the way the VMWare team makes decisions to actively spite its free hypervisor users. Now, I know we aren't their focus (big expensive enterprise Vsphere licenses are) but they often just make my blood boil with their aggravating behavior.

I can deal with stubborn and unfriendly development teams and user communities (after all, I've been using FreeNAS for almost 5 years now) but I struggle more with using a product where it feels like the developer hates me.

 

[rma]

What about xenserver? XenServer | Open Source Server Virtualization

 

[biggen]

^

Xen gives you everything that VMware gives you but for free. Try it with orchestra.

 

[kac77]

Well I've been using KVM for a while now and the main reason I went to it is because the virtualization packages are built into the kernel. You don't have to worry about upgrading and breaking anything. The hardware specifications aren't anywhere near as strict. Basically if it runs on Linux and supports VT-D / IOMMU then it will work...period.

Virt-Manager has come a long way and it pretty much supports everything you need it to. I pair it up with ZFSonLinux and really haven't looked back since. Using Linux as my host also means that I get to use Mythtv to drive my HTPC recording needs.

Probably the only problem that exists is with Windows 10 + certain games. The games are usually the ones that cause the most trouble normally like Batman. Other than that I haven't had any real issues. I use Fedora.


Let me know if you need any help. I can walk you through it.

 

[Zarathustra[H]]

Well I've been using KVM for a while now and the main reason I went to it is because the virtualization packages are built into the kernel. You don't have to worry about upgrading and breaking anything. The hardware specifications aren't anywhere near as strict. Basically if it runs on Linux and supports VT-D / IOMMU then it will work...period.

Virt-Manager has come a long way and it pretty much supports everything you need it to. I pair it up with ZFSonLinux and really haven't looked back since. Using Linux as my host also means that I get to use Mythtv to drive my HTPC recording needs.

Probably the only problem that exists is with Windows 10 + certain games. The games are usually the ones that cause the most trouble normally like Batman. Other than that I haven't had any real issues. I use Fedora.


Let me know if you need any help. I can walk you through it.

Appreciate the info!

I had read that Virt-Manager was something that only masochists would touch, and that the way to do KVM pretty much was Proxmox, but maybe that is on older Virt Manager?

I'm not too concerned with windows or gaming. My virtualized box is a dedicated server that resides in my basement. It has 10 guests on ESXi currently, two of which are BSD based, the rest are Linux.

If I decide to go with Proxmox for ease of management, can I still tweak things with Virt Manager, or is it a one or the other type thing?

 

[Zarathustra[H]]

The more I think about it, I really like Proxmox for its easy integration of containers and VM's.

Since most of my VM's are really just single application isolated linux guests, it would seem that I have an opportunity to save gobs of RAM by switching them over to containers (I'm not concerned about CPU cycles, since my dual hexacores with HT are WAY overkill right now, but I guess that will be a benefit too)

 

[kac77]

The more I think about it, I really like Proxmox for its easy integration of containers and VM's.

Since most of my VM's are really just single application isolated linux guests, it would seem that I have an opportunity to save gobs of RAM by switching them over to containers (I'm not concerned about CPU cycles, since my dual hexacores with HT are WAY overkill right now, but I guess that will be a benefit too)

You really can't go wrong with either one since this is for home use. But you'll save memory regardless on every Linux guest you run. The virtio-balloon driver will reduce/add memory as needed if you over provision guests. Even if you don't my VPN server get's by on 512MB of RAM and actually all of my Linux guests survive on that or less.

 

[Zarathustra[H]]

Proxmox is by far the most complete and easiest to use KVM front end (its actually more than that...). But you might find dealing with the attitude from the "team" at Proxmox just as frustrating as dealing with VMWare.

You really can't go wrong with either one since this is for home use. But you'll save memory regardless on every Linux guest you run. The virtio-balloon driver will reduce/add memory as needed if you over provision guests. Even if you don't my VPN server get's by on 512MB of RAM and actually all of my Linux guests survive on that or less.

Can anyone explain to me the issue of running containers ontop of ZFS?

My plan is to install Proxmox onto two mirrored 500GB SSD's, which will also serve as "datastores" (to use VMWares terminology as I can't think of what else to call it right now), using the ZFS installer on the Proxmox disk.

I have read that containers don't much care for ZFS though, and that it can cause problems. I have also read that there is some sort of ZFS plugin that makes the containers work, but it is unclear to me if I need to create a separate ZFS volume with this plugin, or if the plugin allows the use of the same ZFS volume as the installer.

Can anyone help clear this up?

 

[PigLover]

Can anyone explain to me the issue of running containers ontop of ZFS?

My plan is to install Proxmox onto two mirrored 500GB SSD's, which will also serve as "datastores" (to use VMWares terminology as I can't think of what else to call it right now), using the ZFS installer on the Proxmox disk.

I have read that containers don't much care for ZFS though, and that it can cause problems. I have also read that there is some sort of ZFS plugin that makes the containers work, but it is unclear to me if I need to create a separate ZFS volume with this plugin, or if the plugin allows the use of the same ZFS volume as the installer.

Can anyone help clear this up?

Containers work fine on ZFS. You just have to set the cache mode correctly.