• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ESXi 5.5 GPU Passthrough

K

koei

Gawd
Joined
Feb 7, 2006
Messages
538
Can anyone list a couple current consumer grade GPU that are supported for passthrough? I tried searching everywhere, but found somewhat conflicting information. Some sources indicate most AMD GPUs are supported, while VMware states that only pro grade GPUs are approved.
 
Full passthrough - none.
vDGA/etc - Kepler cards mostly.
What works with passthrough - AMD cards.

IIRC, of course.
 
Check out this website listed in another thread here.

http://www.pugetsystems.com/labs/articles/Multi-headed-VMWare-Gaming-Setup-564/

I've done a LOT of testing with DVGA and SVGA (direct and shared GPU.)

Ultimately, you have two options:
1 - DVGA - direct connect a video card to a VM using PCI Pass through. This will force you to attach a monitor to the video card in your ESX server and pass through a USB port so you can have a keyboard and mouse.... Lame design but you get the fancy benefits of your desktop being a VM.

2 - SVGA - shared graphics where you can run RDP or a client like VMware View (horizon) and have gaming graphics on a thin client. This will require you to buy a very expensive video card and it will complain and only work on special motherboards. I really hate how AMD (not as much as NVidia) and NVidia have made their drivers super picky with the hardware it detects and if it allows you to use the GPU. There are blogs where you can hack a desktop grade video card into a server based card. Long story short, the hack is 100% possible but getting it to work in ESXi and run as SVGA is sketchy. I messed with this and I could get the video card to work great in a full windows install but getting VMware drivers to play nice was difficult. The easy hack was still forcing you to buy a three year old video card for $400-$500.

Every six months I spend some cycles testing what is available on the market to create the "ultimate thin client gaming system." I just finished testing with some AMD FirePro and Nvidia Grid cards with Horizon view 6 and, I feel, we are getting closer but not there yet. *** shake fist! ***


Nicholas Farmer
 
Very informative. Thank you nicholas.

I tried using R7 250E, which is basically a 7750 as DVGA. Passed it through in ESXi, which also showed the device as 7750. However, subsequent reboot to apply the change resulted in bootup freezing.
 
Hi Koei

I would recommend you to read this post, there's a few steps to make it works.
https://communities.vmware.com/thread/297072


You need to start whit 2gb of ram ,

edit the conf

pciHole.start = "1200"
pciHole.end = "2200"

In 5.5 and up , you need to use EFI bios

In my case i had to disable audio in the bios.

Also id recommend to make snapshot before testing the vtd. for easy come back in case it did not work.

* forgot : you need to install vmtools first also
 
Last edited:
Which would be a better performance an amd r9 280 or a Nvidia Quadro 4000? Not the K4000 since that's about 100 dollars more expensive, but the r9 280 and the Nvidia Quadro 4000 are about the same price, anyone know which one I should get? And can I not just use a r9 290? I read through the Multi-headed VMWare Gaming Setup article, and they only mention r9 290x not the r9 290?
 
Maybe I have just been lucky, but I have yet to come across ANY expansion card that doesn't work flawlessly with IOMMU/VT-D based direct I/O forwarding.

As mentioned above though, this means the video card will behave as if it were directly connected to the guest, so any output will be local, only for that guest, via the outputs on the back of the card.

Any remote graphical management would need to be done via VLC or some other 3rd party graphical remote management software.
 
Zarathustra[H];1040983305 said:
Maybe I have just been lucky, but I have yet to come across ANY expansion card that doesn't work flawlessly with IOMMU/VT-D based direct I/O forwarding.

As mentioned above though, this means the video card will behave as if it were directly connected to the guest, so any output will be local, only for that guest, via the outputs on the back of the card.

Any remote graphical management would need to be done via VLC or some other 3rd party graphical remote management software.
Click to expand...



Could you have many guests using IOMMU/VT-D based direct I/O forwarding powered on all at one time? Say 10-30 guests?
 
WoKeN said:
Could you have many guests using IOMMU/VT-D based direct I/O forwarding powered on all at one time? Say 10-30 guests?
Click to expand...

Probably not that many, given the lack of that many PCI-E slots/ devices available to pass through, and the number of IRQs that have to be perma-allocated may be a limit as well at a certain point.
 
lopoetve said:
Probably not that many, given the lack of that many PCI-E slots/ devices available to pass through, and the number of IRQs that have to be perma-allocated may be a limit as well at a certain point.
Click to expand...

What he said.

When you direct i/o forward a device to a guest, that device belongs to that guest, and that guest only. It can not be shared with other guests.

So if you had 30 guests, and each wanted a direct forwarded GPU, you'd need a GPU per guest, and I haven't seen any motherboards lately with that many PCIE slots :p

Also, keep in mind, in order to do Direct I/O forwarding the host needs to support either IOMMU (AMD) or VT-D (Intel).

On official server hardware, this is pretty much universally supported (but still check just to make sure). If you are using consumer hardware for your server it gets trickier.

Many Intel non-K model CPU's support VT-D, but it is very rare to find a consumer motherboard that has it enabled in BIOS, and since it isnt a feature most consumers care about, it usually doesnt e ven show up in official specifications, and if you get it working, you could be only a bios update away from it no longer doing so.

On AMD - on the other hand - IOMMU is supported on pretty much all modern motherboard CPU combinations.
 
Zarathustra[H];1040983699 said:
What he said.

When you direct i/o forward a device to a guest, that device belongs to that guest, and that guest only. It can not be shared with other guests.

So if you had 30 guests, and each wanted a direct forwarded GPU, you'd need a GPU per guest, and I haven't seen any motherboards lately with that many PCIE slots :p

Also, keep in mind, in order to do Direct I/O forwarding the host needs to support either IOMMU (AMD) or VT-D (Intel).

On official server hardware, this is pretty much universally supported (but still check just to make sure). If you are using consumer hardware for your server it gets trickier.

Many Intel non-K model CPU's support VT-D, but it is very rare to find a consumer motherboard that has it enabled in BIOS, and since it isnt a feature most consumers care about, it usually doesnt e ven show up in official specifications, and if you get it working, you could be only a bios update away from it no longer doing so.

On AMD - on the other hand - IOMMU is supported on pretty much all modern motherboard CPU combinations.
Click to expand...


I'm going to be using a x9da7 supermicro mobo and 2 e5-2665s, but I was hoping there was some way I could have 20-30 guests utilizing graphics card, is it not possible at all? What type of graphic support do these guest get if any if they don't get their individual graphics card supporting them?

Basically I'm asking if there's any option to share my graphics card(s) across multiple guests?
 
Last edited:
WoKeN said:
I'm going to be using a x9da7 supermicro mobo and 2 e5-2665s, but I was hoping there was some way I could have 20-30 guests utilizing graphics card, is it not possible at all? What type of graphic support do these guest get if any if they don't get their individual graphics card supporting them?

Basically I'm asking if there's any option to share my graphics card(s) across multiple guests?
Click to expand...

They get a virtualized SVGA card. Remember, ESX was designed for server software - no need for fancy graphics on most of those.

Your only option is VDGA, and it uses specialized video cards and drivers to "carve up" the card for sharing between guests, and those start at around $1500 a card minimum.
 
WoKeN said:
Basically I'm asking if there's any option to share my graphics card(s) across multiple guests?
Click to expand...

You can do that using a NVIDIA GRID K1/K2 and share the GPU to many guest OS. This is through the VMware SGA driver, and you allocate a specific amount of memory per instance. Thus you can share one K1 card between 150+ users, depending on the MB per guest.
 
Shame, I really can't afford to drop 1500-2k on a graphics card for this project right now. Is there anything I can do with hyper-v and windows server either 2008 or 2012 that can do what I'm hoping to achieve? Share graphics card across multiple guests? Or xen?
 
WoKeN said:
Shame, I really can't afford to drop 1500-2k on a graphics card for this project right now. Is there anything I can do with hyper-v and windows server either 2008 or 2012 that can do what I'm hoping to achieve? Share graphics card across multiple guests? Or xen?
Click to expand...

Can you explain to us some details about your project? Then maybe we can make more intelligent recommendations.

Is each guest going to be doing compute stuff on the video card, or why do they need GPU's?
 
I want to run anywhere from 16-30 guests of diablo 3 simultaneously, please don't judge me
 
good luck with that as well - the virtualized 3d card in workstation probably wont run D3 all that well either. :(
 
Then is there any option at all you may recommend? Short of a nvidia grid gpu, could I use nvidia quadro for this purpose?
 
Some quadro cards are compatible with vDGA, but those are also 1500+ for most of them.
 
Okay so I haven't touched VMWare in a while, but I can tell you what these guys are saying up til now, and that avoiding the Grid GPU's is not going to get you a good result. I know those cards are in the range of $3-5K, but they are among the first to actually enable full motion 3D at usable FPS. If you have played with https://www.onlive.com or other streaming gaming sites you will have an idea of what is capable with GRID technology.

Of course there are probably a lot of details that are missing from your scenario like what is the expected connectivity rates of the clients? Is this a LAN setting or Remote? Bandwidth and latency will play into the scenario.

I have to agree that up to this point nVidia is the leader in vGPU technology up til now. You asked the question what is the best solution that can be recommended. To be honest it's not VMWare. XenServer has better support for this type of environment, ESXi is still about a year out to have support you really want.

Citrix has been doing this for over a decade now, and I can recall doing Ultima Online on Windows Terminal Server using the Citrix client pretty good over an ISDN back in 2000. The technology really didn't advance like it should have until recent pushes by technology investors (such as nVidia) which is the result of the GRID technology and vGPU. This has only happened over the past 2 years that type capabilities of full motion 3D and GPU sharing were possible

I hope this makes sense. Yes your K4000 type GPU can do some work, but will not support 30+ without multiple cards (even with GRID). I can see 2-6 maybe on the older technology and it will have lag issues. In either Case XenServer + XenDesktop will yield a better supported platform at this point. Otherwise you will have to wait until sometime in 2015 when VMWare catches up.
 
You must log in or register to reply here.
Back
Top