Migrating a VM from one datastore to another throwing an error.

[AceXsmurF]

I am running vsphere 4.0 foundation (no vmotion) and am trying to migration one VM from a DAS storage to iscsi storage and getting the following error: "moving a virtual machine that has snapshots is not supported when the virtual machine has disks placed outside of its home directory". The VM is shutdown when I am trying to migrate it, just fyi.

From everything I can tell all the disks for this VM are in the folder for this machine. There are a number of snapshots, but I have migrated VM's with snapshots before just fine. Has anyone else ran into this issue, or have any advice?

 

[dasaint]

it is not recommended moving snapshot systems to another datastore due to the possibility of loss of data, in this situation your snapshots are localized within your configuration folder where as your main storage flat is located somewhere else.

your main flat-vmdk is referenced within your snapshots descriptor vmdk

(there are 2 files that make a hard drive vmdk)
a .vmdk -- descriptor file (small in size)
a -flat.vmdk -- main data container (hard drive size)

the descriptor points at the main -flat file with information about it. so say if your data containers are in different locations your snapshot descriptor is going to go down the chain

for example. if you have 5 snapshots and they are sequential they are going to run like this to make the hard drive operate

snapshot 5 -> snapshot 4 -> snapshot 3 -> snapshot 2 -> snapshot 1 -> main flat

if there is a break in the chain like if you were to move data container from one store to another the link from snapshot 1 to the main flat could possibly loose its path information becuase it may try and reference its original location versus its new location.

this is what causes that error to show.

you can move the drive but you must make sure the PID and the CID follow the chain down and point correctly down to the parent. because if they dont the hard drive becomes inaccessable..

best option is to commit your snapshots and then move your data, this is the recommended way of doing this.. there is no reason to be running on snapshots except for updating and using for view based environments. outside of that you shouldnt be running on snaps because they can grow massivly because all they store is RAW data. (example. if you edit a 20meg file 1000 times that raw data is stored at 20x1000 == 20000MB flat file, it takes raw changes and commits the entire file so its very possible to grow massivly and quickly)

hope that helps.

 

[AceXsmurF]

What I ended up doing was deleting all the snapshots and moving the VM. One of my QA guy's cried a bit, but hey at least they are not working on physical machines.

Thanks for the good information if I ever hit this again.

 

[dasaint]

no problem, glad to help

 

[NetJunkie]

I'll just add, don't keep VM snapshots around very long. I usually recommend less than 48 hours. It can take a LONG time to recommit snaps, depending on the write profile of the server. I've seen VMs take 3 days to commit a snapshot and during that time you can't do some things to the VM. I've had more than one customer think it was taking too long to commit a snap so they shutdown the server..and then vCenter won't let them power it back on until the snapshot task is done. Talk about stress...waiting on your SQL box to finish before 8am.

 

[AceXsmurF]

We keep snapshots around for literally many months. I would say 90% of our VM's are for QA/testing of our software. So every VM that we build after it is setup we take a snapshot of a 'clean' setup. Then depending on testing we will take snapshots of various version installations in order to do upgrade testing and other application testing that we regularly go back to.

The production VM's that we are running we really do not touch much, but those also have 'Clean' snapshots that were done after everything was setup and working right, so if something goes wrong we can go back to it. Now that I actually type that out, it just doesn't seem like that good of an idea. We backup the data inside the VM's, but I guess we really should do some backups of the actual VM's that are production VM's.

 

[lopoetve]

We keep snapshots around for literally many months. I would say 90% of our VM's are for QA/testing of our software. So every VM that we build after it is setup we take a snapshot of a 'clean' setup. Then depending on testing we will take snapshots of various version installations in order to do upgrade testing and other application testing that we regularly go back to.

The production VM's that we are running we really do not touch much, but those also have 'Clean' snapshots that were done after everything was setup and working right, so if something goes wrong we can go back to it. Now that I actually type that out, it just doesn't seem like that good of an idea. We backup the data inside the VM's, but I guess we really should do some backups of the actual VM's that are production VM's.

Snapshots don't work like you think they do. Not even close.

You never, ever, ever, want a snapshot on a production VM. Those files are growing like mad and you're losing performance the entire time. They're NOT a copy of the VM. It's a "redo" log.

 

[lopoetve]

I'll just add, don't keep VM snapshots around very long. I usually recommend less than 48 hours. It can take a LONG time to recommit snaps, depending on the write profile of the server. I've seen VMs take 3 days to commit a snapshot and during that time you can't do some things to the VM. I've had more than one customer think it was taking too long to commit a snap so they shutdown the server..and then vCenter won't let them power it back on until the snapshot task is done. Talk about stress...waiting on your SQL box to finish before 8am.

AMEN.

Largest I've seen: 4.2TB snap, 2 weeks to commit.

 

[AceXsmurF]

Snapshots don't work like you think they do. Not even close.

You never, ever, ever, want a snapshot on a production VM. Those files are growing like mad and you're losing performance the entire time. They're NOT a copy of the VM. It's a "redo" log.

Would it be safe to say that if you revert all the time to the snapshot that it is ok to do that?

 

[lopoetve]

Would it be safe to say that if you revert all the time to the snapshot that it is ok to do that?

eh. It makes for a nasty file structure, and you can get in trouble with them, so maybe on a dev box, but I wouldn't on any kind of production machine.

 

[AceXsmurF]

eh. It makes for a nasty file structure, and you can get in trouble with them, so maybe on a dev box, but I wouldn't on any kind of production machine.

Ok, thanks for the advice!

 

[Bookmage]

interesting, I did not know that about ESXi snapshots. So for a development environment, would Virtualbox or something else be better suited for VMs?

or should this be a different post...

 

[NetJunkie]

interesting, I did not know that about ESXi snapshots. So for a development environment, would Virtualbox or something else be better suited for VMs?

or should this be a different post...

For long term snaps your best bet is really doing the snaps on the storage array. Let it do what it was built to do.

 

[lopoetve]

For long term snaps your best bet is really doing the snaps on the storage array. Let it do what it was built to do.

This.

 

[The_Mage18]

For long term snaps your best bet is really doing the snaps on the storage array. Let it do what it was built to do.

+1

Also avoid products that perform backups via snapshots. vRanger works that way. My employer lost an entire month's worth of dictations because vRanger didn't remove a snap and begin committing data again. That was the day we threw it out the door.

The only place we have had success with snaps is on a stand alone Workstation machine. The snaps make it easy to do application repackaging as MSI or virtual apps with Novell Zen App. You can have a clean slate, install the app and package it, then roll back.