Missing data

I'm completely baffled on how this happened. It is a lil bit complicated and long I'll try and explain as much as I can. Everything here is an ESXi VM.

On the server in my sig I have about 14 hard drives spread across 3 different controllers.
The onboard motherboard Sata controller & 2 different LSI 8308ELP I'll refer to them as "Onboard Sata" "LSI 1" "LSI 2"

One day I wake up to network problems. I find that my Win2k12 domain controller is having an issue. I look at the console and it appears to be stuck everytime i try to reboot. It gives me the option to repair. Auto repair fails and i;m too lazy to mess with advanced repair so I'm like screw it guess ill just reinstall it from an template one day. This VM resides on a drive that is attached to OnboardSata.

Later I realize I can't seem to talk to my Win2k12 FileServer. It appears to have the exact same problem the domain controller did. So I'm like UGH maybe that controller had a hiccup and corrupted my VMs. This one resides on OnboardSata as well but a totally different drive.

I check on my PfSense VM. It too is having issues at reboot. It resides on the same harddrive/controller as the domain controller. Okay that makes sense what I was thinking earlier something happened on that controller and we got some kinda corruption, whatever.

I get all 3 of those reinstalled on the same hard drives/controller they were previously on. Mind you I used templates and ISOs that resided on the same drive that the domain controller was on. No issues reading those files making new VMs etc. I thought that was kind of weird because i figured if there was some kinda corruption those would be affected as well, but they arent... both servers and pfsense up and running again, whatever moving on with my life. Time to add the data drives back to the FileServer.

I check my main Win7 VM it has 2 drives. An OS drive and a Program Files drive. The OS is on a SSD attached to OnBoard Sata The program files drive is on another drive also attached to OnBoard Sata. It is just fine. No problems

I check my Cryptocurrency wallet Win7 VM. It resides on the same drive/controller as the FileServer. It has no problems. That is kinda weird since the FileServer had corruption.

Here is where the real problem starts

I go to add all the drives to the FileServer VM. Every single one of those datastores are empty. The datastores still show in ESX, I didn't have to re-add them, They still show the names they always had, it is almost like nothing happened but when you browse them there is nothing except the .locker directory. Capacity shows the same size as Free. It is like they are blank drives.....
Now Im like okay...I can understand if one controller had a fit I guess this could happen, but how could this happen across 3 different controllers.....

Cartoons drives 1 & 2 are on LSI 2..... Empty
TV shows drives 1&2 are on LSI 1... Empty
The Backup Drives, which I purposely put on the OnboardSata drives, just in case one of the controllers died so that they wouldn't go with them....Empty

I'm just baffled on how this could happen. The only logical explanation I can come up with is that someone came by and deleted my datastores, but that seems HIGHLY unlikely. I live by myself. Unless someone hacked me and decided to corrupt a couple VMs and delete the datastores. That doesn't seem likely. How come some of the VMs are fine and others aren't even tho they are on the same drives/controllers. I could see a virus perhaps deleting files from within Windows, but not delete the datastores from ESX.

This really bites. Lost all my data including the backups. I'll live, it just sucks. Does anybody have ANY sort of idea? I'm sorry this was so long and thankyou for reading if you got this far.

Well I have lost hope now on the data magically being there. I did a little research and SSH'd into the host with putty and looked in the directories that way. Even in CLI it shows the directories are empty. If I add a file it can see whatever I added.

Is there a log system I can check in ESXi. perhaps I can see when/how the files were deleted?

Okay well I figured it out.
When I deleted the VM it apparently deletes any datastores associated with the VM. I didn't know this. I thought all it did was delete the datastore the the VM was actually installed to. So upon deletion that took all the datastores with it.
When I think about it, what i really should of did was not even deleted the VM, just reinstall Windows over it. Wasn't thinking straight.
Lesson learned

When you delete a VM it delete's the VM's files..if you tell it to. It never deletes an entire datastore. Or you are confusing terms.

Not meaning to be mean, but I also don't understand what you're saying. A VM won't remove a datastore when you delete the vm from disk (I'm assuming you selected "Delete from Disk" instead of "Remove from Inventory").
The only way I could see that happening is if for some reason the guest's VMDK(s) are the actual datastore(s) as well, which just doesn't sound like a good idea at all.
If I'm understanding your setup properly, you have a ton of drives on a host over multiple controllers. What isn't clear is if these were passed through to the VMs, or if they were datastores that the VMs were stored.
I'd also advise heavily against believing that having a set of drives on a 2nd controller is a secure backup. You still have multiple, major single points of failure that will cause complete data loss.
If you can explain more clearly your full setup, perhaps even with a diagram of sorts, that'd help a little bit.

NetJunkie said:

When you delete a VM it delete's the VM's files..if you tell it to. It never deletes an entire datastore. Or you are confusing terms.

Click to expand...

You beat me to it.

Eulogy said:

Not meaning to be mean, but I also don't understand what you're saying. A VM won't remove a datastore when you delete the vm from disk (I'm assuming you selected "Delete from Disk" instead of "Remove from Inventory").
The only way I could see that happening is if for some reason the guest's VMDK(s) are the actual datastore(s) as well, which just doesn't sound like a good idea at all.
If I'm understanding your setup properly, you have a ton of drives on a host over multiple controllers. What isn't clear is if these were passed through to the VMs, or if they were datastores that the VMs were stored.
I'd also advise heavily against believing that having a set of drives on a 2nd controller is a secure backup. You still have multiple, major single points of failure that will cause complete data loss.
If you can explain more clearly your full setup, perhaps even with a diagram of sorts, that'd help a little bit.

Click to expand...

I'll try that tonight when i get some free time.