Only one config save? ESX 5.0-5.1

B

BLKMGK

n00b
Joined
Oct 26, 2012
Messages
11
I'm encountering something very odd attempting to install ESX on a new machine. When done I need passthru for 2x SAS controllers (M1015) and for a single USB bus. the motherboard is a Tyan S5512GM2NR and I'm on BIOS 2.2 (not the new b) and I'm running a Sandy XEON with 32Gig of RAM. This config has worked for others and my end goal is an unRAID server using 8 drives, the rest being used for ESX. I am booting from a Sandisk USB stick mounted in the onboard USB A port.

The scenario:

Install ESX 5.1, passthru SAS controllers, reboot, it takes. Passthru USB, setting disappears. Headbang. Reload, passthru USB, it takes, reboot. Passthru SAS, it refuses to take!

Load ESX 5.0U1, add hardware all at once. Reboot - success! Add license, reboot, license is gone!

Load ESX 5.0U1, add all hardware and license at once. Success! Try this with 5.1 - no joy I only get USB passed and SAS refuses to add. Huh?! Upgrade the 5.0U1 stick, SAS disappears. :confused:

It's as if there's a permission somehow being set on the stick's config files - I can't explain it and I'm no ESX pro.

I'm about to load up 5.0U1 again and add everything at once in hopes of at least having a working system I can go forward with. I've spent MANY hours on this and just would like to get it further along.

Has anyone seen anything like this before? Ideas?

Also, I've spoken to several folks using this board who had issues using Ivybridge Xeon. PCIE x4 slots apparently give them fits and even the 2.02 firmware has given problems with PCI slots passing thru although thankfully I've not got to do that. Has anyone else seen these issues with Ivy CPUs and this board? How much of an advantage is an Ivy CPU?
 
Okay, in frustration I loaded ESX 5.0U1 and set it up to pass my USB, my SAS controllers, and added my license all in one shot. Rebooted and joy everything was there.

Created a VM, passed in the USB controller, but found no USB device on the second add. Puzzled I rebooted the system and the VM I'd just created and saved isn't listed in inventory!

It's the Twilight Zone! Now I have just moved the USB stick to another port after trying\failing to drop the USB #2 bus and enable the #1 bus. Now that I've moved the stick I have been able to drop #2 and add #1. I only have the two USB controllers. My VM has not reappeared but I've got a suspicion that what controller is enabled and where the boot stick is located has something to do with this. I note that the glowing LED on the stick goes dark after boot which I don't think is normal. I only have the one stick plugged in right now. Going to create anothe rVM and see if this one sticks around after a boot!
 
Okay, I think I've got this figured out. I've gotten some help from another unRAID user who uses both ESX and this board. The answer is interesting IMO and my results are differing from his setup.

What was apparently happening was I was co-opting the USB bus the ESX stick was sitting on. Changes made to the config after this point couldn't be saved and explains why my VMs were disappearing. Why I wasn't getting PSOD I cannot explain.

The board's USB controller #1 appears to support ONE of the USB A ports - the one closest to the PCI covers and the USB pin header that is closest to the slot covers as well as the ports on the rear I/O shield. The #2 controller supports USB A closest to the drive bays and the USB A port that's closest to the drive bays. NOT the same as what was seen on another of these boards but once I was sure the boot stick was being taken away it was just a matter of installing a few times and playing port bingo. Grrr!

Once I knew the correct mapping of the controllers to ports and got the USB sticks separated properly and mapped things began working. I am using controller #1 for passthru which allows the boot stick to be in an A port that's closest to the drive bays. The unRAID key will be placed in the USB A port closest to the rear of the case (closest to PCI covers). The case header will be attached to the portion of the mobo closest to the drive bays that has a header and I can use this for kbrd access. Doing this will prevent my USB kbrd from being plugged into the back panel and I'll be forced to use the front port or IPKVM full time.

Finally, this appears to be working. Hopefully my experience can help another!

P.S. Sticking to 5.0U1 for now. I think I could now get this working on 5.1 but I think I've spent enough on it for now. Any real advantage to 5.1 over 5.0U1?
 
Last edited:
definitely a downside to booting off usb if your going to use the server for usb passthrough. I think im just going to try a usb card for passthrough.
 
I never understood what the advantage to booting esx off of a USB stick? The only thing I could come up with is during upgrade less worry of losing any vms that might reside the datastore with esx.
 
Honestly, I boot it off of a stick because it's "easy" and this was how I was taught to do it. Obviously a PITA if there's an issue like the one I encountered and I came very close to booting it from a disk out of sheer frustration towards the end! This is how I've setup other ESX and thus was why I did this one this way - far from an expert obviously. Using a separate card for passthru sounds like a good idea. I'd need one that was PCIE 1x though and didn't spend any time researching them. If there's one out there that's decent I might consider it just to have additional ports available for passthru if I need them. for now I'm just happy the silly thing is working :D I've got some things to workout before I consider this "production" and turn off my other standalone boxes but I'll get there and be learning the whole way which is half the reaosn to do it...
 
You must log in or register to reply here.
Back
Top