VMDirectPath restrictions?

D

danswartz

2[H]4U
Joined
Feb 25, 2011
Messages
3,715
A curiosity bump question here: can anyone explain one of the restrictions? I understand why vmotion and HA are not usable, but am curious as to why the full memory specification needs to be reserved. Not a complaint, I'm an OS engineer, and am just curious what there is about the implementation that necessitates this.
 
Last edited:
That's a very good question, and I'll admit that I don't totally know the answer. I expect it may have to do with the lack of memory swapping that will happen with that, but I'll see if I can find out.
 
So far, the answers have been:
"Goats and runestones"
"Virgins and full moons around the stanford campus"

Still working on it.
 
That was my initial theory, so far doesn't seem to be panning out. Current working theories are TPS issues and / or DMA/IRQ mapping and isolation. Still working. It honestly is not something anyone has asked before...
 
I would have guessed something to do with DMAR. Honestly, I didn't really expect you to research this! Like I said, curiousity bump :)
 
Because a PCI card has a brain of its own that can access the memory controller outside hypervisor's supervision.

When a CPU instruction from a VM calls for an access to some memory location that isn't there an exception will be raised, the hypervisor will intercept that and try to figure out what's supposed to be there. If a PCI card does that the exception will be on the PCI card where there is no hypervisor, so it's just going to crash. Therefore the memory must be there.
 
levk said:
Because a PCI card has a brain of its own that can access the memory controller outside hypervisor's supervision.

When a CPU instruction from a VM calls for an access to some memory location that isn't there an exception will be raised, the hypervisor will intercept that and try to figure out what's supposed to be there. If a PCI card does that the exception will be on the PCI card where there is no hypervisor, so it's just going to crash. Therefore the memory must be there.
Click to expand...

That goes along with what lopoetve said - sounds reasonable to me!
 
levk said:
Because a PCI card has a brain of its own that can access the memory controller outside hypervisor's supervision.

When a CPU instruction from a VM calls for an access to some memory location that isn't there an exception will be raised, the hypervisor will intercept that and try to figure out what's supposed to be there. If a PCI card does that the exception will be on the PCI card where there is no hypervisor, so it's just going to crash. Therefore the memory must be there.
Click to expand...

certainly along the theory we were working on - any details of how you know this? :)
 
That's a good point. I keep forgetting that ram can be paged under esxi, correct? So any memory being used for DMA needs to be pinned for the guest OS?
 
It can't tell which (or if) memory is being used by a card though so it must pin everything down just in case it could be.

Every virtualization solution that does passthrough has the same restrictions and usually have comments to this effect in the manuals or in source.
 
You must log in or register to reply here.
Back
Top