• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

VMDirectPath for HTPC?

nitrobass24

[H]ard|DCer of the Month - December 2009
2FA
Joined
Apr 7, 2006
Messages
10,477
Was thinking about taking the HTPC in my Home Theater and using it as an additional host.
My only thing would be the need for a Win7 VM with HDMI output to my TV.

Is something like this possible? I dont really care about HA or anything for that VM.
 
HA as in Hardware Acceleration, or as in High Availability? (I hate it when the same acronym is used when talking about PCs)
 
VMDirectPath with video cards with ESXi 4/5 is IMHO at this time an exercise in futility. I have seen a couple of successes listed on the ESXi 4 forum @ VMware ... but the vast majority, myself included, have given up on getting it to work at this time.
 
Its possible, but there's a lot of caveats, possibly very buggy, and only been proven somewhat work with a few specific (ATi/AMD I think) cards. You have to be careful how you treat the virtual video card driver otherwise you lose console view(unless you don't care about it). It also has some weird limitations on the amount of RAM you can allocate to the VM.

If you don't care about any of that and want to play around with it, ultimately it IS possible(no support and no guarantees). Here is a good thread you should read thoroughly if still interested. :)

http://communities.vmware.com/thread/297072?start=0&tstart=0
 
Here's a summary of my experience over the past few months.

Video is pretty much the only obstacle--everything else works perfectly well in a virtual environment. That said, video is a big obstacle.

1. Nvidia doesn't work. In the thread over at VMWare one person said he got a Quadro working. Color me skeptical. He may very well have gotten it to work, but until we see posts from others to corroborate I have my doubts.

2. Only some Radeon card/mobo combinations seem to work. I had success with an old HD3450 and partial success with an HD5450 (card worked but DXVA video acceleration wasn't, which made it pretty much worthless). I haven't tried any other radeons but I think people have had some success with some other models.

3. Even though the HD3450 passed-through OK and DXVA seemed to work, video performance (especially at 1080p) was sketchy and I never got HDMI audio playing without a million pops and clicks.

4. The bigger issue (at least for Win7 Media Center with CableCard tuners) was that HDCP (copy protection) would not work, despite it being supported by the card, cable and monitor. I think this has something to do with the VMware virtual video adapter being present and also being the "primary" video adapter. I never found a way to disable the VMware adapter (though the guy who got the nvidia card working said he was also able to get the VMware adapter disabled). Without HDCP and non-glitchy HDMI audio I gave up for now.

I still use a VM for my HTPC and it works great, but I use Xbox 360s as WMC extenders at all my TVs in order to watch live and recorded TV. It works and is reasonably fast but I miss the smooth response of the "native" WMC interface. At some point I figure it will be possible to get a video card passed through such that its full capabilities can be used, but I just got tired of fighting with it. VMDirectPath is very poorly documented, so you're mostly on your own trying to get it working. VMware has a PDF with some troubleshooting info (google VMDirectPath Troubleshooting should pick it up) so if you feel like playing around with it there are some options you can set in the vmx to try to get it working better.

Oh, and if you want to assign more than 2816MB of RAM to the VM then you'll need pcihole.start and pcihole.end values in the vmx. The big thread @ Vmware's forums has details on that.

http://communities.vmware.com/message/1668788
 
I went through all of the suggestions on the mentioned VMware thread with 2 different MBs (Asus P8B WS & Tyan S5510) with 3 different Nvidia cards & 2 ATI cards to no avail. Did the pcihole thing. Could get ATI cards to show up in Device Manager with driver supposedly loaded but I could never get the card to show up in the "screen resolution" screen. Would also get BSOD with both ATI & Nvidia cards. Finally just gave up on the idea for now. Will give it a go again if someone figures out the magic settings (if they exist) to do this.
 
I have actually been trying for the past few days to accomplish the same thing.

I'm running an x3 720 on a 990FXA-UD3 with ESX 5.
I have tried with a Radeon 4830 and X600, both PCIe.
VM running Win 7 Ultimate 64.

I have had basically the same experience as emc. Devices show up in device manager with drivers installed, but no additional displays under screen resolution. Once I install the VM tools or ATI drivers, the VM will BSOD on reboot. System restore to before the VM tools install gets me back to a working vm.

I also tried the pciHole.start and end values with no luck.

I have not given up yet and will spend some more time on it this weekend. I'll let y'all know any successes I have.

Good luck
 
deltronik ... you might want to take a look at the latest post over on the VMDP/ATI topic @VMware. & see if the instructions that BAM279 posted will work. Who knows??? I am no longer set up to do any more testing on this.

Best of Luck!
 
emc, thanks for the heads up, I probably wouldn't have looked back at that thread this weekend as i've already read forward and back.
 
no BSOD on catalyst install or restart!
mouse is acting wonky using console, jumping between screens...

emc, start budgeting some parts, this may end up working :cool:

edit: last update for a while, radeon 4830 is now main and only display on vm! trying to connect a usb mouse now..
 
Last edited:
I have it setup with drivers and vmware tools installed but no second monitor yet.
 
This is exciting news indeed ... I had been trying on Intel C204&6 based MBs. I do have a Gigabyte 790FXA - UD3 MB but that does not have enough PCIe slots for my use. Look forward to all of the details & have fingers crossed that your setup works like a charm!
 
If any of you guys are planning on using WMC with cablecard tuners I'd really appreciate hearing if you're successful passing the Digital Cable Advisor (which is the WMC app that enables WMC for CableCard). I only know of one other person who has tried and neither of us could pass it due to lack of HDCP (despite card, monitor, and cable all being HDCP-compliant). I'm using an extender for now but would love to get the native interface working.

It's still great having an HTPC VM though. My ESXi server is now the only box running 24/7 in my house, doing all sorts of things--really cool!
 
My rig is now a fully functioning htpc... sorry luv2chill, no hdcp.

h/w specs:

Phenom II x3 @ 2.8 (o/c soon, it used to hit 3.2+ on stock volts)
Gigabyte GA990FXA-UD3
2x 4GB 1333 Microcenter DDR3
Radeon x600 128MB PCIe (console)
Radeon 4830 512MB PCIe (flashed higher clocks years ago... don't remember clocks)
Realtek 8168 Gigabit Ethernet (surprised this actually works with esx)
2x Intel EXPI9301CTBLK PCIe (got two before I realized the realtek worked, but they are supposedly much better)
acomdata SuperSpeed USB 3.0 PCIe for mouse/keyboard (could not get either on board controller working with directpath)
ENVY24PT PCI Sound Card for optical out (couldnt get onboard working, had this from my old htpc, 10 bucks from Microcenter :D)

vm:

2vCPU
4GB RAM fully allocated
Win 7 64
Radeon 4830 via VMDirectPath
ENVY24PT via VMDirectPath
acomdata USB 3.0 via VMDirectPath


Getting the 4830 working was tricky. It's apparently all about persistence with the Drivers, though I still can't get CCC to install.
First I added the pciHole.start and pciHole.end values at 1200 and 2200 respectively, but I still got BSODs at boot.

The tip came last weekend from BAM279 on the VMware forums which emc mentioned: http://communities.vmware.com/message/1853298#1853298
I did not get a BSOD during driver install, but the vm did reboot during install.

Once the monitor came up on the 4830 I was able to easily use the resolution settings to disable the VMware adapter.

I have successfully played Dirt 3 and Driver San Francisco at 1080p with no issues.
Actually, at first, Dirt 3 was crashing at the end of races and hiccuping.
I changed the pciHole.end value to 1700 to match the video card's RAM (512MB) and voila! no more hiccups or crashes! Played using wireless XBOX 360 controller passed through by ESX (not VMDirectPath) without any noticeable lag. :cool:

The next issue was the mouse and keyboard as ESX cannot pass through an HID usb device to a vm.
I was unable to get the onboard usb controllers working with DirectPath.
The device showed up in device manager and drivers successfully installed, but no usb devices worked.
I ended up picking up the acomdata USB 3.0 PCIe from MC and it worked perfectly!
It's based on an NEC controller.


The onboard sound wasn't passing through either, luckily I had an ENVY24PT Multi-channel PCI card from my old build. Worked without a hitch. DTS and Dolby Digital working perfect.


Finally, I tried to go to youtube the other day. Installed flash, restarted browser, go to youtube, browser hangs after a few seconds, IE or FF wtf!?
The fix is to turn off hardware acceleration in flash, but my browser hangs within a few seconds, not time to load a site...right click...settings...uncheck hardware acceleration.

I had to re-enable the VMware display, disable the 4830 display, then turn off accleration using the console.


This brings me to what I consider a fully functioning HTPC!

Also on my ESX I have virtualized my UNRAID server and setup a print server.
I'll be using this setup as a test lab for work related stuff too, but I've got I bit more networking to do first.

Hope this helps somebody out...:D
 
deltronik .... You are a Prince among men! Good work ... now I will have to give it a go again. I did buy another ASUS P8B WS in anticipation of trying again. Got it up & running last night with a Xeon E3-1230 & 16 GB of ECC Ram. I will be trying with a AMD 5670 based card. I have a couple of different Nvidia cards as well that I may give a try "just to see".

Again Thank you for the persistance!
 
Ha, thank you for the kind words good sir!

But honestly, emc, I was close to throwing in the towel before you let me know to look back at that thread... It was perfect timing.:cool:

Let us know how your build goes.
 
I'm so bummed ... no joy yet again. Same outcome as before ... get driver installed no problem ... try to swith display to new monitor BSOD.

Ugh!
 
So emc, you are installing the drivers after you've disabled the radeon in safe mode right?
If you are getting BSOD on boot after installing drivers go back into safe mode and disable the card in device manager again and reboot...

Install VMtools
Reboot
Safe Mode
Disable PCIe card in device manager
reboot
Install drivers
PC will crash/BSOD/reboot or reboot the pc manually.
enable PCIe card in device manager if it isn't automatically enabled
Enable second display in resolution properties
extend displays, making second main display
disable 1st (vmware) display

Keep trying the same steps over, as far as I've seen it should work.
The Catalyst drivers always bomb when they get to the vision engine or whatever it is.

Good luck
 
Last edited:
deltronik ... will go back and clean the slate & start over. I have never had a BSOD on boot only when trying to make 2nd display primary. Thank you so much for the detailed step by step that worked for you. It may very well be a BIOS/MB issue. I have been trying on an Intel C206 based board ... I recall you were using an AMD MB. As I mentioned before I do have a 970 based Gigabyte MB so if I cant get the C206 MB (ASUS P8B WS) to work maybe I'll give the AMD one a go.

Thanx again!
 
I can report that VT-d works perfectly with OSX... Radeon 6450 card.

Win7 is a different story.
 
Anyone have anything new to report on this? I have been seriously considering trying to consolidate my existing HTPC into a new ZFS file server build, but if HDCP can't happen then that is a bummer (I'd like to watch blu-rays).

And for Deltronik (or anyone else who made this work):
Is the video smooth? Do you get any kind of jerks or stutters when other VMs are doing a lot of work? I can't have that happening :)

And did you happen to try analog audio or just the optical out?

And why did you need to pass USB in via directpath, can't you just plug in your devices to the onboard then 'assign' them to your VM?

Thanks!
 
I haven't tried anything different since spending a month on this back in the late summer. Unfortunately, it's still such a niche that I haven't read about anyone else attempting to get HDCP working either.

The only thing I've read that looked like it was worth trying was using Xen instead of ESXi. I think someone posted that with Xen they had no problem disabling the virtual graphics card--my hunch is that is what is preventing the HDCP checks from passing.

I also had hardware acceleration issues and HDMI audio crackles and pops, but I didn't spend a whole lot of time troubleshooting those as the lack of HDCP made the whole endeavor pretty much a non-starter. The HTPC VM works perfectly for me, but that's because I'm only using it as a server and use Xbox 360s for the front-end. I'd suggest that as a possible solution for you too, but that won't help you with Blu-Ray.

Hope you get some more responses. I'd still love to get it working as the media center extender interface on the Xbox is pretty laggy in comparison and imposes several limits on the types of files that can play.
 
Back
Top