Hypervisor preference fight - GO!

[cyclone3d]

Moving this discussion here from a different thread.

My preferences are:

1. For VM servers - Hyper-V on Server 2025 Datacenter
Reasons:
A. Windows Server VMs do not require a license (unlimited)
B. Clustering / Failover works very well once configured properly.
C. Just about as quick as bare metal...maybe a tiny bit behind ESXi
D. Networking is dead simple to set up.

2. For local VMs - VMWare Workstation Pro
Reasons:
A. Free
B. Way more mature than VirtualBox - faster, less buggy, and just works.
C. USB Device mounting actually finally works properly
D. Shared folder with host is easy to set up and use.
E. Doesn't delete the virtual drive like ESXi does when deleting a VM.
F. Broadcom ruined ESXi with their stupid pricing.

Questions:
VirtualBox
1. Has it been fixed to actually work properly with an XP VM? Last time I tried it, it had major issues with color and resolution settings and would just lock up / stop responding.
2. Has performance been fixed? It has been super slow compared to ESXi, Hyper-V and ProxMox.

Proxmox
1. Has the issues with syncing between clustered servers been fixed? When I was using it not too long ago, if one server lost connection or was rebooted, it was a huge tossup of whether or not the heartbeat and thus syncing / failover would show back up properly. This was with a cluster of 3 identical servers.

 

[LodeRunner]

I run a Windows 2019 DC failover cluster with StarWind VSAN at home, and a larger identical cluster with a hardware SAN at work.

A lot of my workloads are Windows (RDS Server, AD DC (DNS and DHCP), Exchange for giggles (on prem at work has been shut down after hybrid conversion).

I run Linux workloads, mostly Ubuntu but some RHEL, and they perform fine.

I kinda wish I had SCOM:VMM as it was the MS sort of equivalent to the vSphere management appliance. But my network isn’t sober complex so it’s not hard to create the network config and replicate it to the other hosts.

For work, the DC pricing just made sense, we have enough Windows VMs for various reasons that buying standard and then the necessary additional licenses would have cost more than getting DC for the hosts.

 

[CrimsonKnight13]

Moving this discussion here from a different thread.

My preferences are:

1. For VM servers - Hyper-V on Server 2025 Datacenter
Reasons:
A. Windows Server VMs do not require a license (unlimited)
B. Clustering / Failover works very well once configured properly.
C. Just about as quick as bare metal...maybe a tiny bit behind ESXi
D. Networking is dead simple to set up.

2. For local VMs - VMWare Workstation Pro
Reasons:
A. Free
B. Way more mature than VirtualBox - faster, less buggy, and just works.
C. USB Device mounting actually finally works properly
D. Shared folder with host is easy to set up and use.
E. Doesn't delete the virtual drive like ESXi does when deleting a VM.
F. Broadcom ruined ESXi with their stupid pricing.

Questions:
VirtualBox
1. Has it been fixed to actually work properly with an XP VM? Last time I tried it, it had major issues with color and resolution settings and would just lock up / stop responding.
2. Has performance been fixed? It has been super slow compared to ESXi, Hyper-V and ProxMox.

Proxmox
1. Has the issues with syncing between clustered servers been fixed? When I was using it not too long ago, if one server lost connection or was rebooted, it was a huge tossup of whether or not the heartbeat and thus syncing / failover would show back up properly. This was with a cluster of 3 identical servers.

My preferences:

Proxmox VE 9.x - network hosted baremetal virtualization running on Debian 13
Easily upgradable & updatable - custom kernels & modules can be utilized
Free & no restrictions on updates
QEMU/KVM virtualization allows for a large range of OSes
Clustering & failover work as expected despite past issues with such (Ceph & HA)
ZFS migration for VMs works perfectly between hosts
Networking works out the gate once bonds & bridges are setup
VFIO (PCIe passthrough) for GPUs & NICs works for every VM assigned these devices
GPU acceleration with virtual & passthrough for VMs
HTML5 web interface is amongst the best for hypervisors, with both the local hosts & clusters
Advanced configurations of hardware settings with serial console support through a host's terminal
ARM & ARM64 emulated virtualization - actually works
USB passthrough for VMs through host & VM reboots
VM deletion can include or not include virtual disks removal
Remote terminal administration through SSH
LXC & OCI container hosting for less resource utilization & near baremetal experience
Docker deployments work great through VMs or LXC containers
VLAN support & security capabilities
Can support macOS VMs like ESXi & doesn't require any advanced modification
SPICE video streaming support for easy console administration via GUI

VirtualBox - local VMs on CachyOS (Arch Linux)
Free
Easily updated
Not buggy & slow as described above
Shared folder hosting very easy
Doesn't delete disks if wanted
Runs as fast as KVM (one of many Linux virtualization technologies) works on the local host, which isn't slow at all for a higher end system
Full 3D acceleration on VMs
USB passthrough

Answers:
VirtualBox
1. Never tried running XP on it since it's a legacy Windows OS that I only play with on PVE9 randomly
2. Yes, it's not slow at all. If you haven't used it in a long time & did so on a slow system, that's a very outdated view.

Proxmox VE
1. That is no longer an issue with 9.x. I suggest testing it out for yourself.

 

[Hash_Immutable_1337x]

For local VM's : I love Hyper-V role versus other workstation products e.g. "VMware Workstation", "Virtual Box", etc..
Primarily the feature that works well is the Dynamic Memory feature... When testing out various Windows Server VM's... you can really squeeze a large amount of VM's on your workstation if your working through some lab scenarios, etc..
( in addition if you have worked with Windows for a while.. Hyper-V is so f'ing easy to use )

Barely breaking a sweat

 

[cyclone3d]

For local VM's : I love Hyper-V role versus other workstation products e.g. "VMware Workstation", "Virtual Box", etc..
Primarily the feature that works well is the Dynamic Memory feature... When testing out various Windows Server VM's... you can really squeeze a large amount of VM's on your workstation if your working through some lab scenarios, etc..
( in addition if you have worked with Windows for a while.. Hyper-V is so f'ing easy to use )

Barely breaking a sweat
View attachment 821113

The reason I don't like Hyper-V as a workstation product is because it sticks the host OS on top of the hypervisor and ends up slowing down the host.

 

[kram182]

I like Hyper-V for pretty much everything but I'm jealous of VMWare's better vGPU/3D performance, especially for older/XP/etc systems (GPU passthrough negates this with Gen2 Hyper-V etc)

 

[LodeRunner]

I was contemplating a Proxmox cluster with StarWind VSAN in their new appliance VM, but the NVMe drives I bought to go in my R640 nodes apparently will randomly freak out and fail into diagnostic mode when Linux is running. They don't do this with Windows. No one has a good answer as to why. HUSMR7638BDP3Y1, EMC branded drives, and no one seems to have firmware for them that might fix it. So I guess I'll be doing soft-raid across the NVMe drives or a bunch of multiple iSCSI targets in StarWind for Windows instead of the CVM VM (the only free option is their RHEL container designed for Proxmox which requires certain scripts to be run from the PVE host to modify the disk image in order to run properly). TrueNAS Scale likewise caused the drives to fail into diagnostic mode.

I keep waffling on just asking them for a quote, but I'm sure it's more than I want to pay just for my home lab.

 

[CrimsonKnight13]

I was contemplating a Proxmox cluster with StarWind VSAN in their new appliance VM, but the NVMe drives I bought to go in my R640 nodes apparently will randomly freak out and fail into diagnostic mode when Linux is running. They don't do this with Windows. No one has a good answer as to why. HUSMR7638BDP3Y1, EMC branded drives, and no one seems to have firmware for them that might fix it. So I guess I'll be doing soft-raid across the NVMe drives or a bunch of multiple iSCSI targets in StarWind for Windows instead of the CVM VM (the only free option is their RHEL container designed for Proxmox which requires certain scripts to be run from the PVE host to modify the disk image in order to run properly). TrueNAS Scale likewise caused the drives to fail into diagnostic mode.

I keep waffling on just asking them for a quote, but I'm sure it's more than I want to pay just for my home lab.

You might need to test the following with the kernel cmdline options:

pcie_aspm=off nvme_core.default_ps_max_latency_us=0

 

[LodeRunner]

You might need to test the following with the kernel cmdline options:

pcie_aspm=off nvme_core.default_ps_max_latency_us=0

I'll give that another poke at some point; redownloaded the VSAN CVM and am going to see if my free license works on it or not. Their documentation is sometimes not the best. But VSAN running as a Windows service (Samsung PCIe HHHL 3.2TB and 40Gbit networking in both hosts) has been stable and quite good performance on my current R720 nodes. The R640's haven't yet been setup to replace them. The CVM does ZFS and other things which drew my interest the first go around.

Proxmox does make it easier to pass PCIe devices in than Hyper-V (interface-wise).