Zarathustra[H]
Extremely [H]
- Joined
- Oct 29, 2000
- Messages
- 38,857
Hey all,
So here is my scenario: I have a few cheap hacked Chromeboxes as HTPC's in the house. They are great. Unfortunately Intel's Linux EGL + VAAPI video decoode drivers are causing me all kinds of headaches, and I really wan to move away from integrated Intel graphics, but I can't with the Chromeboxes, which won't accept GPU:s (no expansion slots in that tiny form factor)
Since I already have a ESXi server running way under capacity (CPU load average: 0.15, 0.13, 0.12) the somewhat complicated solution is as follows:
Replace all my HTPC's with virtual machine guests, each with a passed through GPU, an HDMI over Ethernet extender and a USB over Ethernet extender.
I figure this will both save power (the host is on all the time anyway, so I will just be using the extra power when active) and eliminate having to deal with multiple physical boxes around the house, and manage everything from one central location.
This could be a fun project, and should be cheaper per HTPC, than buying new cases, power supplies, motherboards, etc. etc, even the low end types needed for a HTPC.
At first I wanted to use Geforce GT 720's as they sip power, are single slot width, and are PCIe 8x, and fit nicely into my Supermicro X8DTE server boards 8x slots without modification.
Unfortunately, Nvidia - in their wisdom - blocks GPU passthrough on consumer boards. While AMD passthrough generally works, the VAAPI support in Linux has been flaky, just like with Linux, The whole point of this project is to get a set of nice Nvidia VDPAU supporting GPU's
After some research, the Quadro/Tesla/GRID driver release notes have a list of compatible GPU's for pass through. The cheapest ones on eBay right now are the Quadro 2000's, essentially a downclocked GTS 450. I lost a few auctions for boards that sold used for $20-$30 (I suck at that last minute bid thing everyone does) but eventually wound up with one for just under $40. I figure I'll start with one as an experiment and add one or two more later as I bid on decent ones on eBay and win them cheaply.
They are 16x PCIe though, so I'll need to take a dremel to them to make them 8x compatible. Also they are only VDPAU feature set C, so no 4k or HEVC support, but that is OK for now. By the time I go 4k or HEVC in a big way, there might be cheap GPU's used that can do that, and be passed through, but those definitely don't exist now at a reasonable price.
My planned parts list for each HTPC guest is thus as follows:
Would appreciate your comments and this project with any thoughts.
I understand USB passthrough of Keybaords andMice can be tricky in ESXi, but according to this Vmware KB I should be able to do so after manually editing the .vmx to include the following lines:
Hopefully this will do the trick.
What would be even cooler would be if I could pass through the USB hub. That way I can treat the USB hub as if it were a local machine, unplugging and plugging back in the devices of my choice, without repeatedly going back in and forwarding the devices.
By default HID devices and hubs are filtered out, but since there is a config line (like the above) to unfilter HID devices, is there also a config line to unfilter hubs?
I know, I could just get a USB controller and pass that through, but with all these GPU's going in, PCIe slots are not plentiful, so that isn't an option.
Another potential concern (Which I have discussed in this thread) is that x8 slots are generally 25W, (16x slots are 75W) and the Quadro's can draw up to 62.5W at full load. My server board has two 8 pin EPS connectors, and is using low power CPU's though, so I am thinking it will probably work without putting too much of a load on the 24pin connector. Besides, the video cards will only ever see idle loads or video decoee loads, never full 3d rendering or compute loads, so they might just stay in that 25W window on their own...
Hoping for some good feedback on this one
So here is my scenario: I have a few cheap hacked Chromeboxes as HTPC's in the house. They are great. Unfortunately Intel's Linux EGL + VAAPI video decoode drivers are causing me all kinds of headaches, and I really wan to move away from integrated Intel graphics, but I can't with the Chromeboxes, which won't accept GPU:s (no expansion slots in that tiny form factor)
Since I already have a ESXi server running way under capacity (CPU load average: 0.15, 0.13, 0.12) the somewhat complicated solution is as follows:
Replace all my HTPC's with virtual machine guests, each with a passed through GPU, an HDMI over Ethernet extender and a USB over Ethernet extender.
I figure this will both save power (the host is on all the time anyway, so I will just be using the extra power when active) and eliminate having to deal with multiple physical boxes around the house, and manage everything from one central location.
This could be a fun project, and should be cheaper per HTPC, than buying new cases, power supplies, motherboards, etc. etc, even the low end types needed for a HTPC.
At first I wanted to use Geforce GT 720's as they sip power, are single slot width, and are PCIe 8x, and fit nicely into my Supermicro X8DTE server boards 8x slots without modification.
Unfortunately, Nvidia - in their wisdom - blocks GPU passthrough on consumer boards. While AMD passthrough generally works, the VAAPI support in Linux has been flaky, just like with Linux, The whole point of this project is to get a set of nice Nvidia VDPAU supporting GPU's
After some research, the Quadro/Tesla/GRID driver release notes have a list of compatible GPU's for pass through. The cheapest ones on eBay right now are the Quadro 2000's, essentially a downclocked GTS 450. I lost a few auctions for boards that sold used for $20-$30 (I suck at that last minute bid thing everyone does) but eventually wound up with one for just under $40. I figure I'll start with one as an experiment and add one or two more later as I bid on decent ones on eBay and win them cheaply.
They are 16x PCIe though, so I'll need to take a dremel to them to make them 8x compatible. Also they are only VDPAU feature set C, so no 4k or HEVC support, but that is OK for now. By the time I go 4k or HEVC in a big way, there might be cheap GPU's used that can do that, and be passed through, but those definitely don't exist now at a reasonable price.
My planned parts list for each HTPC guest is thus as follows:
- Nvidia Quadro 2000 GPU
- Monoprice HDBaseT Extender Kit
- Monoprice USB over Ethernet Extender
- Basic 2 port USB hub for the TV side so I can connect both my Logitech K400 Keyboard/Touchpad and my Windows Media Center Remote.
- Displayport to HDMI with Audio adapter
- Lots of cables
Would appreciate your comments and this project with any thoughts.
I understand USB passthrough of Keybaords andMice can be tricky in ESXi, but according to this Vmware KB I should be able to do so after manually editing the .vmx to include the following lines:
Code:
usb.generic.allowHID = "TRUE"
usb.generic.allowLastHID = "TRUE"
Hopefully this will do the trick.
What would be even cooler would be if I could pass through the USB hub. That way I can treat the USB hub as if it were a local machine, unplugging and plugging back in the devices of my choice, without repeatedly going back in and forwarding the devices.
By default HID devices and hubs are filtered out, but since there is a config line (like the above) to unfilter HID devices, is there also a config line to unfilter hubs?
I know, I could just get a USB controller and pass that through, but with all these GPU's going in, PCIe slots are not plentiful, so that isn't an option.
Another potential concern (Which I have discussed in this thread) is that x8 slots are generally 25W, (16x slots are 75W) and the Quadro's can draw up to 62.5W at full load. My server board has two 8 pin EPS connectors, and is using low power CPU's though, so I am thinking it will probably work without putting too much of a load on the 24pin connector. Besides, the video cards will only ever see idle loads or video decoee loads, never full 3d rendering or compute loads, so they might just stay in that 25W window on their own...
Hoping for some good feedback on this one
As an Amazon Associate, HardForum may earn from qualifying purchases.