• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ESXi 5 Passthrough / VMDirectPath

D

dmg210

n00b
Joined
Dec 13, 2011
Messages
34
Hi Guys,

This is my first post here but hopefully someone can help.

I'm trying to configure VMDirectPath on ESXi 5 with a Dell Perc 5e / 6i storage controller.

Hardware Specs:
SuperMicro x8sil-f
Xeon 3440
32gb ram
Dell Perc5e with 8 x 1tb Samsung F3's (raid 5) - pass-through to Windows Server 2008 R2 for CIFS (doesn't pass-through!!)
Dell Perc6i with 8 x 2tb Samsung F4's (raid 5) - pass-through to Windows Server 2008 R2 (need firmware 6.3.1-0003+)
Dell Perc6i with 4 x 120gb OCZ Agility SSD (raid 0) for VMFS

ASUS ATI Radeon HD 4350 - pass-though to XBMC for HDMI out with Audio (testing in progress...) - HDMI over ethernet + USB over ethernet for remote control

My first problem is when trying to pass-through the Perc 5e. I can configure the device for passthough, reboot the ESX host and add at it to a virtual machine. It is also detected by the OS and I'm able to see the controller in device manager and the LSI storage configuration manager.

The problem occurs when trying to create a NTFS partition and format the array. The Windows disk manager either hangs or the OS bluescreens...

With the Dell Perc6i I'm able to see the controller in device manager, it shows up as an LSI Logic 1078, although I doubt this is correct as a remember seeing both an LSI storage controller + Dell Perc 6 controller on another system. I'm unable to see any disks in computer manager and the LSI storage software is unable to detect the controller from Windows.

I've also had the ESX host purple screen of death when powering on the VM with the Perc6i configured in passthough mode. I'll post the error log later (PCPU error) i think.

Can anyone confirm they have successfully managed to pass-through a Dell Perc 5e/6i?
 
Last edited:
psod.png


When trying to passthrough the Perc6i the ESX host crashes, its running on Dell firmware v6.2.13

I've found another post where someone has successfully passed though a Dell Perc6/i integrated, is there any different between the integrated / adapter versions?

http://hardforum.com/showthread.php?t=1628818#post1037625828

I can only see the LSI adapter, sas 3000 series adapter in device manager not the Perc 6.

Any ideas?
 
Did you setup the array and initialize it in the Perc setup?

in the ESXi health/hardware status does everything look normal and all the drives show up?


Also you could be having heat problems. Do you have a HSF on the Perc?

This is mine

mL4kg.jpg
 
Yeah I have a little fan on each like yours :)

Everything shows up correctly in the ESXi health / hardware monitor, i'm just in the process of trying ESXi 4.1 but I'm having to wipe one of my datastores and format back to VMFS 3.

On the Perc5e I created and initialized the RAID from the Perc BIOS, I'm able to successfully write to the array using an Acronis boot disk.

The Perc6i array is already full of data that I would like to move ASAP so I don't want to change the firmware on this card just yet. I'm happy to try LSI firmware on the PERC 5e but which firmware should I use? Is it the same as the Perc5i (LSI MegaRAID SAS 8480E) firmware?

Thanks
 
Last edited:
Well I've just realized the Perc 5e is connected via the PCI-E bridge, this would explain the PSOD i guess. I'm surprised this hasn't been fixed in ESXi 5.0.

The devices show up differently in ESX 4.1 but I'm still unable to see the Perc6i in device manager, just the LSI adapter.

esx41.png


perc6i.png
 
Actually I'm really quite confused.

According the diagram in the MB manual there are no PCI-E bridges on this board.

I've tried the Perc5e card in all three slots with no success, I'm running out of ideas now...

I'm beginning to wonder if this board even compatible with VMDirectPath?
 
Last edited:
from looking at the dump, it's totally fucked the PCI bus.

Double check bios/firmware, but I expect you're done.
 
I'm not sure, could be that you have the Perc 5i and I'm trying to use to Perc 5e.

Its running latest Dell firmware but its very old 12/9/2008. I've just tried the LSI firmware from the megaraid sas 8480e but that hasn't worked. When booting natively into Windows the device cannot start with LSI firmware. Hope a haven't killed it...
 
dmg210 said:
I'm not sure, could be that you have the Perc 5i and I'm trying to use to Perc 5e.

Its running latest Dell firmware but its very old 12/9/2008. I've just tried the LSI firmware from the megaraid sas 8480e but that hasn't worked. When booting natively into Windows the device cannot start with LSI firmware. Hope a haven't killed it...
Click to expand...

I use the Perc 6.
 
Arhh yeah sorry, Well I've not had any luck with the Perc6i either. Do you think there is a different between the adapter / integrated version of the card?

I've found someone else who seems to be having the same problem as me:

http://vmware-forum.de/viewtopic.php?t=23731

Do you think he means the MB was faulty? I can't make sense on it...
 
mmm i can tell you i have been doing ESX 4/4.1/5.0 with PCI Passthrough on a LSI-1068e card, also have a buddy who is doing the same with a couple of them and we havent had one single problem with them, i have been running rock solid for a long time with these setups.. (Nexenta Core/Stor OS and i have also used OpenIndianna with it)

PCI Passthrough can be problematic with the wrong card from what i hear, also i think its still an experimental technology.

Make sure you have IOMMU and if available VT-D enabled on your motherboard that could also be a part of the problem.
 
Are you using the IBM ServeRAID BR10i (LSI-1068e) by any chance?

I guess your running ZFS on Nexenta core/Stor OS? Whats the performance like?
 
Can anyone confirm successful pass-through of any kind of device PCI-E with the SuperMicro X8SIL-F

This must be possible surely?
 
Last edited:
dmg210 said:
Can anyone confirm successful pass-through of any kind of device PCI-E with the SuperMicro x8sil-f

This must be possible surely?
Click to expand...

Yes, i did on mine. I was using the Perc 6 with the X8SIL-F.

I would try flashing the BIOS and FIRMWARE.
 
I tried using the latest LSI firmware 7.0.1-0083 on the Perc5e with the same result, note I was running Dells latest firmware 5.2.2-0076.

The BIOS is up to date, running R1.1 - unless there's a beta I don't know about?

I'm just backing up the data on the Perc6 then I'll try updated from 6.2.13 to the latest version.

I don't suppose you know which versions your running do you?
 
dmg210 said:
Can anyone confirm successful pass-through of any kind of device PCI-E with the SuperMicro X8SIL-F

This must be possible surely?
Click to expand...

i use it with a SuperMicro AOC-USAS-L8i in my home all-in-one
- no problem yet - so it must be a problem with the perc or its firmware

only thing. If your X8 Bios is quite old, try to update
 
Yeah it works fine on a regular install of Server 2008.

The card hasn't been modified and were purchased from Dell a few years ago.

The "pin mod" shouldn't be required with the SuperMicro X8SIL-F anyway as the SM BUS can be disabled with a jumper on the board as it is by default.

Thanks for idea though.
 
dmg210 said:
Yeah it works fine on a regular install of Server 2008.

The card hasn't been modified and were purchased from Dell a few years ago.

The "pin mod" shouldn't be required with the SuperMicro X8SIL-F anyway as the SM BUS can be disabled with a jumper on the board as it is by default.

Thanks for idea though.
Click to expand...

I was thinking if it was modded it could cause the problems.

Did you try ESXi 4.1?

Also just a though, try using the Dell ESXI ISO?
 
Yep tried both ESXi4.1 / 4 with the same results.

I did try the Dell ESX ISO a little while ago but it will need to be modified to run on my system as it checks the MB vendor details to validate if its a Dell system or not. Might be worth another try though.
 
Just a thought, try to disable the on-board RAID/SATA controller and any other PCIe devices you can from the BIOS.
 
I'll give that a shot...

Are you using are a hardware RAID setup in your system with VMDirectPath or something else?

Apart from enabling VT-D in the BIOS is there anything else you had to change?
 
Also try putting it in the x4 slot, should be the 3rd one down. That comes off the Northbridge according to the Block Diagram in the manual.
 
Thanks for taking a look at the manual, do you think there's a bridge or something on the first two slots that's not shown in the manual?

Do you have yours in the third slot then?
 
dmg210 said:
Thanks for taking a look at the manual, do you think there's a bridge or something on the first two slots that's not shown in the manual?

Do you have yours in the third slot then?
Click to expand...

No they are directly connected to the CPU. lopoetve said it is messing up the PCI bus so maybe moving it there will help?
 
You are the man, AMD_GAMER!!! thanks

I've upgraded the firmware on my Perc 6i to 6.3.1-0003 and placed the card in slot 2 (x8) which has successfully passed though to server 2008! Just running some tests now but I can see the whole array and I've just moved 100gig or so without any issues!
 
Last edited:
Do you think a pair of these with the Perc6i and an external enclosure would work?

irsa-pm1.jpg


Any idea what the pins are for?
 
Last edited:
No idea... supposedly, it should work, but I've never tried something like that.

Glad the firmware worked :)
 
After a little more research I can confirm the Perc5e card is incompatible with ESXi pass-through due to the onboard PCI-X to PCI-E bridge! That's why I was seeing a PCI bridge when passing through the controller.
 
I've been trying to get a Perc 6/i (discrete adaptor) to work for a few weeks now without success, how did you do this?

(I realise this is an old thread but I'm very stuck!)
 
trif55 said:
I've been trying to get a Perc 6/i (discrete adaptor) to work for a few weeks now without success, how did you do this?

(I realise this is an old thread but I'm very stuck!)
Click to expand...

Just looking back at the old thread as I couldn't really remember...

What system board do you have? When passing through gpus it seems the pci slot is important. Also a combination of firmware & drivers can play a part... What stage are you at?
 
You must log in or register to reply here.
Back
Top