KVM under SmartOS or OpenIndiana?

[DarkNovaNick]

I'm looking to set up a server to store files using ZFS and also run some virtual machines. The napp-it all in one concept is basically what I'm looking at doing, but I am a bit disappointed by the fact that with ESXi you need VT-D support in order to do it properly, as VT-D support limits your choices of hardware significantly.

I read that this past summer Joylent ported KVM to Solaris under their SmartOS project and it was also put into OpenIndiana.

Since these OS's run ZFS natively and can host virtual machines using KVM, it should work to run either SmartOS or OpenIndiana on the bare metal, with the function of handling ZFS, hosting the virtual machines with KVM, and optionally using napp-it and doing file server duty. The advantage I see is that less complex hardware is required when compared to ESXi. You need a recent processor like an i3, i5, i7 to do KVM, but that's it, you don't need a specific motherboard or a SATA card or anything.

Has anyone tried doing this, or read about it and decided not to do it for some reason they'd like to share?

 

[unhappy_mage]

I'm looking to set up a server to store files using ZFS and also run some virtual machines. The napp-it all in one concept is basically what I'm looking at doing, but I am a bit disappointed by the fact that with ESXi you need VT-D support in order to do it properly, as VT-D support limits your choices of hardware significantly.

I read that this past summer Joylent ported KVM to Solaris under their SmartOS project and it was also put into OpenIndiana.

Since these OS's run ZFS natively and can host virtual machines using KVM, it should work to run either SmartOS or OpenIndiana on the bare metal, with the function of handling ZFS, hosting the virtual machines with KVM, and optionally using napp-it and doing file server duty. The advantage I see is that less complex hardware is required when compared to ESXi. You need a recent processor like an i3, i5, i7 to do KVM, but that's it, you don't need a specific motherboard or a SATA card or anything.

Has anyone tried doing this, or read about it and decided not to do it for some reason they'd like to share?

KVM can also make use of VT-D, and the i3/5/7 processors support it anyways, so I'm not sure what distinction you're trying to draw there. The cards you need for PCI passthrough in ESXi/KVM are the best cards to use anyways. Use the LSI 2008 series cards, like the 9211 or the IBM M1015.

KVM seems like a great idea, but I have AMD hardware, so I can't use it on my machine. Therefore, I've gone with ESXi. Honestly, I wish I could use it, but Joyent apparently doesn't have any AMD hardware.

 

[DarkNovaNick]

KVM can also make use of VT-D, and the i3/5/7 processors support it anyways, so I'm not sure what distinction you're trying to draw there. The cards you need for PCI passthrough in ESXi/KVM are the best cards to use anyways. Use the LSI 2008 series cards, like the 9211 or the IBM M1015.

Yes, sorry I wasn't clear, my concern with VT-D is even though these processors support it, you need a motherboard that supports it and also a card like LSI 2008. With KVM on something like SmartOS you could theoretically use any motherboard and even the onboard SATA ports if need be.

Its not so much a concern for the main system -- I can buy these parts. I'm thinking more for backups -- what if I build an ESXi system and the motherboard fries? Or the SATA card dies? I will need spares of all these parts or I won't be able to bring up the system. I'm thinking that with SmartOS I could just move the drives over to any other system that has a recent processor and bring things up while I'm waiting for a replacement part to come in. It seems like that's one of the great things about virtual systems -- if the hypervisor supports your hardware then it is easy to move things around to a different physical server. But I value stability above all and wouldn't want to use something like KVM under StormOS if its not ready for production use.

 

[Nnyan]

Just a note but VT-X is not VT-D. There are a limited number of reasonable processors but you are very restricted on your motherboard choices. I am also going to give SmartOS a try even though my server is on ESXi 5 compat list.