VT-d and folding in a VM?

A

Activate: AMD

[H]ard|Gawd
Joined
Nov 6, 2004
Messages
1,994
I'm looking at putting some new hardware into my current home server rig, and I'm toying around with the idea of making it a folding box. The inital goal when building was compact size and low power, so I was using an extra s775 board with an underclocked CPU. However, the thought occurs that I'd like to switch to running a ZFS-based OS running in a VM using ESXi 4.1. That means using an SB cpu for the VT-d capability, and it also got me thinking that I could run linux in a VM and fold at the same time. Unfortunately, the k SKU's don't have VT-d :rolleyes:, only the non-k and xeons, so I'd probably go with a xeon server board (aka no OC'ing). Does anyone have a feeling about the feasibility of doing this and what kind of PPD i'd get to see? I was thinking that because of VT-d's ability to give more direct control over subsystems to the host OS it might have less of an impact on PPD, especially since the media serving aspect gets used only intermittantly. I know that somebody here is running some SB xeons and they do excellent PPD/watt, so I'm definitely interested in trying this out if its at all feasible
 
VT-d is more for things like VMDirectPath to do things like pass through PCIe cards to guests rather than to simply setup CPU folding VMs.
 
pjkenned said:
VT-d is more for things like VMDirectPath to do things like pass through PCIe cards to guests rather than to simply setup CPU folding VMs.
Click to expand...
right, which is why I want to use it with my SAS card for the ZFS VM. If i'm getting a new rig, I'd like it to be able to fold, which limits my OS choices to windows or linux UNLESS I use a VM. I guess the question is not 'is VT-d is some kind of magical way of doing effective folding in a VM', but rather what kind of PPD i'd see in this usage case (assume a CPU like an E3-1230), and does VT-d have any impact either way
 
An i5 @3.2GHz is good for 11-12K ppd on Windows, limux will improve on that. Running kinux in a VM is very close to running Linux natively.

H.
 
Ah I did not read that paragraph fully. My bad, the performance hit is not too bad. One thing is that things like ZFS deduplication and such can use a lot of CPU so your ZFS VM is likely going to be a bigger drag on overall PPD than using the VM.
 
VT-d shouldn't make folding perform any better than a CPU with VT-x, unless you plan on folding on GPUs as well. In other words, it would be just like folding in a VM on any other type of CPU, so you'd take a bit of a performance hit, but it would still be worth it.
 
Zero82z said:
VT-d shouldn't make folding perform any better than a CPU with VT-x, unless you plan on folding on GPUs as well. In other words, it would be just like folding in a VM on any other type of CPU, so you'd take a bit of a performance hit, but it would still be worth it.
Click to expand...

pjkenned said:
Ah I did not read that paragraph fully. My bad, the performance hit is not too bad. One thing is that things like ZFS deduplication and such can use a lot of CPU so your ZFS VM is likely going to be a bigger drag on overall PPD than using the VM.
Click to expand...

cool. thanks for the feedback :cool: I'll have to crunch some accounting numbers first and decide whether I should do this or put together a dedicated OC'd SB rig. obviously ppd/$ will be better for the latter, but the space footprint is a big consideration. I'm hoping that the ZFS CPU penalty would be less of an issue since its a very low use system (just hosting movies, music and doing backups) with few reads or writes.
 
I am very interested if you get this working and what your performance hit is. If you can document what you do that would be terrific. I bought a Xeon E1230 last month and was looking into using some variant of Solaris for ZFS (currently running ubuntu server) and probably running a VM or ESX-I so I can crunch at the same time. I haven't had anytime to set it up or even try to.
 
Haitch said:
An i5 @3.2GHz is good for 11-12K ppd on Windows, limux will improve on that. Running kinux in a VM is very close to running Linux natively.

H.
Click to expand...

Is there a difference significant enough to make one want to run a Linux VM versus sticking with Windows?
 
tjmagneto said:
Is there a difference significant enough to make one want to run a Linux VM versus sticking with Windows?
Click to expand...

It varies depending on your system - most single proc systems see about a 10-20% PPD increase, my dual hex core Opteron gets 60%+ more PPD. Thats running natively, but my Linux VM on a sandy bridge gets higher PPD than it did under Windows, will dig out the numbers.

H.
 
Haitch said:
It varies depending on your system - most single proc systems see about a 10-20% PPD increase, my dual hex core Opteron gets 60%+ more PPD. Thats running natively, but my Linux VM on a sandy bridge gets higher PPD than it did under Windows, will dig out the numbers.

H.
Click to expand...

The system in question would be the 980x system in my signature. Thanks for checking on it.
 
That means using an SB cpu for the VT-d capability
Click to expand...

I have posted this before in another thread:

It's contrary, even though Intel places the attribute to the processor, it is the core logic (chipset) that supports VT-d not the processor. Remember this is an IOMMU feature; not part of the MMU of the processor.

Also, the chipset may have support for VT-d, but the BIOS may not enable it due to OEM's support. Also the kernel, OS, and VMM may not be properly configured or compiled to support it.


Technical notes: http://software.intel.com/en-us/art...-for-efficient-virtualization-of-io-devices// "The VT-d DMA-remapping hardware logic in the chipset sits between the DMA capable peripheral I/O devices and the computer’s physical memory"


http://www.intel.com/support/processors/xeon5k/sb/CS-031637.htm


To note, if it needs to be part of the processor, it is interesting that Intel does not even note in the Ark page the capable attribute for some processors; including my Core i7-940. However, I can enabled support in the motherboard's BIOS for VT-d. This is one thing that Intel is really pissing me off with lately, and making this disconcerting to the end user is not a good thing.


Edit: Although, I would not be surprised if this confusion finally ends and it will be an option in the BIOS related to the processor as so: http://www.anandtech.com/show/4285/westmereex-intels-flagship-benchmarked/2 Look at the BIOS options for Vt-d which seem to be all related to the processor now, instead of the core logic.
 
Last edited by a moderator:
Shikami said:
To note, if it needs to be part of the processor, it is interesting that Intel does not even note in the Ark page the capable attribute for some processors; including my Core i7-940. However, I can enabled support in the motherboard's BIOS for VT-d. This is one thing that Intel is really pissing me off with lately, and making this disconcerting to the end user is not a good thing.


Edit: Although, I would not be surprised if this confusion finally ends and it will be an option in the BIOS related to the processor as so: http://www.anandtech.com/show/4285/westmereex-intels-flagship-benchmarked/2 Look at the BIOS options for Vt-d which seem to be all related to the processor now, instead of the core logic.
Click to expand...
So then why is it explicitly not listed for some processors, specifically the K's? I assume that if its part of the bios, the bios code probably "detects" it based on the cpu microcode. Do you mean that boards capable of utilizing VT-d on any chip would be capable of enabling it on processors that aren't listed as being VT-d enabled? Or do you need to use a hacked bios? It would be nice to have VT-d capability on a 2600k/z68 combo.

Sorry if its off topic, I hesitate to make this a discussion about VT-d since this isn't the place for it, just as it pertains to folding.
 
So then why is it explicitly not listed for some processors, specifically the K's?
Click to expand...

I cannot say, and it is very irritating because Intel seems intent on making this a disconcerting issues for many interested in this feature. . All that I can say is that the processor does VT-x not VT-d. VT-d is related to the core logic used on the motherboard (IOMMU). Even though the chipset may be able to do VT-d, it may not even be enabled/supported in the BIOS and the software/hardware may not support it also.

The only speculation I can offer is that even though the core logic handles the IOMMU not the processor, the option can only be available if the processor "supporting" it is in installed. Meaning that option becomes "available" even though the system is already capable. I think this is how Intel is handling the situation now (at least with Sandy Bridge). Rather than straightly acknowledging it within the core logic as it previously was.
 
Activate: AMD said:
I'm looking at putting some new hardware into my current home server rig, and I'm toying around with the idea of making it a folding box. The inital goal when building was compact size and low power, so I was using an extra s775 board with an underclocked CPU. However, the thought occurs that I'd like to switch to running a ZFS-based OS running in a VM using ESXi 4.1. That means using an SB cpu for the VT-d capability, and it also got me thinking that I could run linux in a VM and fold at the same time. Unfortunately, the k SKU's don't have VT-d :rolleyes:, only the non-k and xeons, so I'd probably go with a xeon server board (aka no OC'ing). Does anyone have a feeling about the feasibility of doing this and what kind of PPD i'd get to see? I was thinking that because of VT-d's ability to give more direct control over subsystems to the host OS it might have less of an impact on PPD, especially since the media serving aspect gets used only intermittantly. I know that somebody here is running some SB xeons and they do excellent PPD/watt, so I'm definitely interested in trying this out if its at all feasible
Click to expand...

Why not run Sol11Express on the bare metal?
 
You must log in or register to reply here.
Back
Top