Guest went missing - ESX 5.5u1

Z

Zetro

Limp Gawd
Joined
Dec 2, 2005
Messages
296
Hey everyone,

I have run into an odd issue that VMware themselves has not been helpful on. Here is my support ticket to them:

"On October 2nd a guest named WDNS001 was hosted on our VMware 5.5 Cluster specifically located on Host 3 of that cluster. At around 11AM we lost all access to this guest.

I have spent a significant amount of effort reviewing all Tasks/Events for that period, pulled the vpxd and vpxd-profiler logs and reviewed them. There is no record of a task showing the deletion or anything related to the server on October 2nd.

I also reviewed the data stores and was unable to find any data related to the VM.

For us to continue to have confidence in our infrastructure I need to understand where this guest is. Having worked with VMware products since ESX 3.0 I have never seen something on this nature prior."

VMware asked for the same logs that I reviewed and is unable to determine the cause. Any insight on where I should be looking?

I am at a bit of a loss on this one :(
 
I have never personally seen a VM disappear from a datastore unless it was a storage related issue or someone deleted it from the datastore. If that VM is in fact no longer on the datastore, it probably isn't a VMware issue.
 
I agree with your logic but I am just surprised that I don't see any mention of not seeing the VM or anything in the logs or an associated alarm in VMware.

I will hunt through the NetApp now but obviously I am not very confident that I will be able to find anything on it :(
 
Even if it were removed from the NFS datastore, wouldn't vCenter show it as orphaned? I may be wrong about it, but I thought that orphaning was how esxi handled VMs in inventory but not in datastore.

Seems like a really odd case... to be removed from a datastore and vCenter without a trace.
 
Eulogy said:
Even if it were removed from the NFS datastore, wouldn't vCenter show it as orphaned? I may be wrong about it, but I thought that orphaning was how esxi handled VMs in inventory but not in datastore.

Seems like a really odd case... to be removed from a datastore and vCenter without a trace.
Click to expand...

This was my thinking as well. If someone removed the device from inventory after it was orphaned I would see the event in the vpxd logs
 
I'm not too familiar with netapp. Is there a way to view the NFS logs on the netapp to see if anything else besides the ESX host accessed the NFS mount?
 
Eulogy said:
Even if it were removed from the NFS datastore, wouldn't vCenter show it as orphaned? I may be wrong about it, but I thought that orphaning was how esxi handled VMs in inventory but not in datastore.

Seems like a really odd case... to be removed from a datastore and vCenter without a trace.
Click to expand...

This is correct. If it's not in vCenter either, it was removed cleanly from ~somewhere~.

On the host in question, you want the hostd.log files - grep for the name of the VM, and if it was actually deleted you should see the call being made to hostd in there. Or it being powered off, at the very least (vmkernel should have teh same message for poweroff, but it doesn't care about deletions).
 
@Gea

That NFS bug is related to APD type issues, it wouldnt just delete the VM out of vCenter nor delete the VM out of the datastore, plus it would show an APD condition in the VMware logs which would have been mentioned... Nothing wrong with updating just dont think that would have led to this issue...

VMs don't go missing randomly... I agree with LOP check the hostd.logs and syslogs if you have them to see what might have happened to the VM.
 
You must log in or register to reply here.
Back
Top