An error occurred while saving the snapshot

[opoet3]

Hi

When i want to create snapshot on ESXi 7.0.2, 17867351 i get this error.

An error occurred while saving the snapshot: msg.changetracker.MIRRORCOPYSTATUS. An error occurred while taking a snapshot: msg.changetracker.MIRRORCOPYSTATUS.

How can i solve this error?

 

[kenobi79]

hi

verify if this kb of veeam talk about your problem

https://www.veeam.com/kb4003

 

[opoet3]

Thanks

This is not related to my problem.

[eode]

Hi!

Got the same error today, basically triggered from Veeam, but related to taking snapshot of my VM, before backup.

Finding
I did a quick troubleshoot in the vmware.log of the affected VM, and found a locking issue, like this:
"2021-05-16T15:38:49.953Z| vmx| | W003: DISKLIB-CBT : ChangeTrackerESX_GetMirrorCopyProgress: Failed to copy mirror: Lost previously held disk lock".

Resolution/workaround
Since the locking is held locally by the host, I just did a vMotion of the VM (to another host, in my cluster), to re-issue the lock by another host. Re-trying snapshot, it now completed successfully (and backup is now working again).

[opoet3]

Thanks.

I had reached this solution before, but this solution is a temporary solution, and after a few days the error is repeated for other virtual machines, and by vmotion them to other hosts, the error is repeated for other virtual machines.

[eode]

Interesting, I'll have to wait to see if the same pattern happens here. Backup went fine yesterday, and this night (as usual). Only occured on one (1) VM for me, so far.

 

- I did also notice a DRS vMotion (auto) occured on the VM earlier the same day (some hours before I got the snapshot/locking issues).

Q: Maybe you could check, if same pattern occured in your environment; e.g. vMotions/changes on the VM before snapshot issues occured? If so, lowering the DRS threshold could also help "slow down" the occurence (not a permanent solution, but the impact/frequency may be reduced).

 

I'm also running ESXi 7.0.2 build-17867351, in a 4-node vSAN cluster.

Q: Are you also running vSAN, or is this regular VMFS/NFS datastore?

[opoet3]

I disabled DRS a long time ago but it didn't have much effect, our infrastructure uses a VMFS datastore.

[nachogonzalez]

Do you see any consolidation errors on the VM?
What happens if you open the Snapshot manager?

[opoet3]

No consolidation error.

Snapshot manager is empty.

[continuum]

1. check your Veeam VM wether it still has mounted vmdks that should be not mounted.
2. check your Windows VMs wether they report VSS-problems - and deal with the errors.

[opoet3]

Thanks.

1. No.

2. No.

[NathanosBlightc]

Please search for more details of possible causes in the vmware.log file via running the following command, especially when you try to generate the snapshot:

cat  /vmfs/volumes/[datastore-name]/[vm-name]/vmware.log

You can also mix the command with the (  | grep -i "snapshot ) to search for related keywords

[eode]

If you see my first reply, I'm getting error with disk lock, and from the process DISKLIB-CBT. Workaround to clear lock for me; migrate the VM to another host. I have not had any issues with locking since. May be issues with CBT/CTK, which also may be disabled on VM level (if only occuring on some VMs, etc.). May be other underlying issues with hosts, etc.

Previous post
https://community.broadcom.com/vmware-cloud-foundation/discussion/an-error-occurred-while-saving-the-snapshot-1#bmc564768e-e76f-42e3-a6ad-253a45a452ca

The error I'm referring to

2021-05-16T15:38:49.953Z| vmx| | W003: DISKLIB-CBT   : ChangeTrackerESX_GetMirrorCopyProgress: Failed to copy mirror: Lost previously held disk lock

 

Since this process fail, the following snapshot request process (SnapshotPrepareTakeDoneCB) will also fail, hence the error seen in vSphere Client (my guess)

2021-05-16T15:38:49.976Z| vmx| | I005: SNAPSHOT: SnapshotPrepareTakeDoneCB: Failed to prepare block track.
2021-05-16T15:38:49.976Z| vmx| | I005: SNAPSHOT: SnapshotPrepareTakeDoneCB: Prepare phase complete (Could not get mirror copy status).

 

I wrote a short guest post, which includes the full vmware.log for a VM, while having snapshot issues, and trying to trigger a manual snapshot (regular snapshot, via vSphere Client), which includes some more details. The relevant part is of course the warning from DISKLIB-CBT, regarding the lock.

https://vninja.net/2021/05/18/error-occurred-while-saving-snapshot-msg.changetracker.mirrorcopystatus

 

I would start with verifying if it's the same error (see the same error in the vmware.log), then go from there. If Production w/SnS, just open a SR with VMware, OFC.

[opoet3]

VM log:

[opoet3]

VM Log:

Moderator note by wila: Pulled your post from the spam queue and moved the log details into a spoiler.

 

[Mullen Ladd]

Reduce the number of search terms. Each term you use focuses the search further. · Check your spelling. A single misspelled or incorrectly typed.

 

Official website

[opoet3]

UP