Snapshot files are not deleted in the datastore

[Goodfred]

Hello everyone!

We have the actual versions of ESXi and vCenter.

I saw that after backups with Acronis-backup that the snapshot files are not deleted in the data-store after the job.
After a backup there is 1 more entry (2 files, hostname-000030.vmdk and hostname-000030-ctk.vmdk with incremental filenames) in the datastore.
Then I tried to create a manual snapshopt. Then there was 1 more entry. After deletion of the snapshot the entry remains in the datastore.

vCenter shows me an information/a warning on this machine: "Die Festplatten der virutellen Maschine müssen konsolidiert werden" ("the hard-discs of the virtual machine have to be consolidated)

Rightclick on VM, "Snapshots" and "Konsolidieren" ("consolidate") doesnt work. The files still remain in the datastore.

I hope that you can help me here!  
I'm thankful for answers, wish a good day and stay with kind regards
Goodfred

[continuum]

Your vmsd-file is invalid - nothing unusual after Veeam or Acronis-automatic backups.
Suggestion: power off VM.
Rename /delete vmsd-file so that it is out of the way.
Create manual snapshot. Then use "delete all snapshots" function.
That normally will result in a clean snapshot chain ...

Tip: vmsd-files are useless when they are invalid - that means that we you see that snapshotmanager is misbehaving very likely the vmsd is corrupt.
A corrupt vmsd file has no good influence at all - in case you are unsure .... "einfach in die Tonne damit"

Ulli

 

 

 

 

 

 

 

 

[Goodfred]

Following happened/I did

1 Power off VM
2 Delete vmsd-file
3 Create snapshot (with powered off VM)
4 Delete all snapshots

Effekt: Not the wanted effekt. The warning and the vmdk files are still here

Then I did

5 Delete again vmsd-file
6 Press "consolidate snapshots"

Effekt: The warning that the hard disc needs to be consolidated is not more present, but the vmdk files are still there.

I'll try later again to create a snapshot and then I press delete-all again. I hope it will work.

Thank you!

[Goodfred]

Now I've selected again the VM and now there are 2 instead 1 "consolidation" messages. (screenshots)

I will try again this steps!

[Goodfred]

I tried additional deletion from the vmsd file between the steps.
Then I got this error: (screenshot)
process failed!
Name: of task: remove all snapshots
Target: my VM
Status: A general systemerror occured: Fault cause: vim.fault.GenericVmConfigFault

Later I did the same without error.

The files still are there and vCenter still says that the VM needs consolidation.

Another question: are you sure that its because of Acronis backup? I've already contacted the Acronis support and they said it is on VMware side/I should proove 1st VMware side. I will contact them again and ask how I can unlock the harddisc from the Acronis-backup-tool. (and I hope with that I can delete the files/successful consolidate the files)

(I'm asking because the files are generated, too, when I add a manual snapshot. And this files are not deleted, too, when I delete the snapshots. Could it be in here anyway that Acronis-backup-tool is the problem in here?)

[a_p_]

Don't get me wrong, I'm not trying to fingerpoint, but only try to find out what may be the issue here.

Is it a physical, or virtual backup server?
How are the backup jobs configured (API, or Hot-Add)?

In case of  virtual backup server, or with backup proxies in place, check each of these backup VMs' settings, to find out if one of them has the VM's (the one with the snapshot) virtual disk mounted as an additional virtual disk. If yes, remove it from the backup server's/proxy's settings.

If it's a standalone backup server, you may reboot it to ensure that it doesn't lock any the VM's .vmdk files.

André

[Goodfred]

It is an Acronis-cloud-backup into the Acronis cloud.

I've installed an Acronis-agent on ESXi level in a VM on ESXi.
(this agent takes care of that backups of the VMs of the ESXi are processed)

There is the option, too, to install agent on VMs/machines directly. With this agents I can backup the single VM/the single machine.

Additional to the ESXi-Acronis-aplliance we have on the VM a Acronis-backup-Plesk-extension - which is an backup-agent, too.
This means - for backuping the server on ESXi-level/VM-level we have the ESXi Acronis-agent.
- for backuping the Plesk instance we have installed on the Plesk-server/VM an additional Acronis-agent.

I will restart now the ESXi-appliance and I hope the steps will work, then.
To restart the agent on Plesk-level I don't have to reboot the server. It's still offline at the moment. (I've noticed at the moment that the Plesk-agent should not be the problem because the machine is powered off )

[Goodfred]

The reboot of the Acronis-appliance did not help.

Do you have any more ideas what I can do?

[a_p_]

I still think that it could be realted to some .vmdk mapping.

What may be an option, is to use RVTools, export the results to an Excel file, and search it for the virtual disk/snapshot in question.

André

[Goodfred]

I've downloaded RVTools.

The export to an Excel file shows the same as the tool itself I think.

I can't find any useful information in the tool/Excel file.

Do you have other ideas what I can do?

Thank you!

[Goodfred]

I've installed a new machine which I wanted to use for the purpose of the machine before.

But now I have the same issue on the new machine!

Now I have to fix it - I can't use another fresh machine.

Do you have any ideas how I can fix this?

Its a fresh installation of Ubuntu 20.04 with Plesk. Backup is done with Acronis Cloud. I have other Ubuntu 20.04 machines (some of them are fresh, too) which dont have this issue and are backuped by Acronis, too

I'm very thankful for answers!

Goodfred

[continuum]

We are making zero progress here ! We still have no clue of what goes wrong and we dont even know what you think is unusual in your case.
May I request a german speaking problem description at least 3 times as long as your earlier posts ?

Ulli

[Goodfred]

Hi!

Wie gesagt, wir hatten eine Ubuntu 20.04 Server VM aufgesetzt und auf dieser Plesk installiert.

Es gibt eine Acronis Agent VM im ESXi die dafür zuständig ist das die ESXi VMs in die Acronis Cloud gesichert werden.
(wie gesagt, durch den Agent kann man alle VMs im ESXi sichern ohne auf den einzelnen VMs einen Agent installieren zu müssen/etc.)

Diese Maschine produziert nach jedem Backup ein Snapshot das leider nicht automatisch gelöscht wird.
Wenn man ein Snapshot manuell erstellt und es wieder löscht, bleiben die Files trotzdem noch.

VMware meldet in 2 Meldungen "Status "Virtuelle Maschine benötigt Konsolidierung" und "Die Festplatten der virtuellen Maschine müssen konsolidiert werden"
Wenn ich auf Konsolidieren klicke gibt es keinen Effekt, er meint aber erfolgreich abgeschlossen haben. (das gleiche wenn man auf "Alle Snapshots löschen" drückt - wohlgemerkt - es gibt keine Snapshots auf die man zurückkehren kann)

Da ich dafür keine Lösung hatte habe ich eine neue Ubuntu 20.04 Plesk Maschine aufgesetzt die aber leider seit paar Tagen (ca. 12 Tage lief es ohne Probleme) die gleiche Meldung und den gleichen Fehler bringt.

Leider weiß ich nicht woran das liegt. Ich hoffte mit der frischen Installation sei der Fehler weg. Da ich nun aber bei der neuen Maschine den gleichen Fehler habe muss ich wohl eine Möglichkeit finden den Fehler zu beheben.

Danke schon mal für deine Antwort!
Grüße Goodfred

[continuum]

Meld dich doch mal per skype -- dann schau ich mir das mal genau an.
Am besten zwischen 11:00 und Mitternacht ...

Gruss aus dem Sauerland
Ulli

 

[Goodfred]

Habe dir eine PN geschickt.

Grüße zurück aus dem Schwabenland!

[Goodfred]

Hello again everyone!

With the assistance from continuum the problem is fixed!

We used the command "vmkfstools -i" to create a new .vmdk file which replaced the multiple snapshot-files. (and the old base.vmdk)

After that the issue didnt appear again! We still only have one .vmdk file after the backup job complete successfully. 

Again a big thank you to continuum!

Kind regards
Goodfred

[a_p_]

If Ulli's steps don't resolve the issue, then there's a good chance that the Acronis backup server (or a proxy) still has one of the VM's virtual disk mounted (in case hot-add is in use). In this case verify that the backup job is done, and remove (don't delete) the hot-added virtual disk from the backup server's configuration.

André