Disk consolidation for VM has failed: Unable to access file since it is locked

[erikberndt]

Last evening we restarted receiving the error message on one of our VM's that Disk consolidation for VM has failed: Unable to access file since it is locked

Disk consolidation fails with the same error message. I checked and there are no snapshots for this particular VM, although we do use VEEAM as a backup solution.

So far I have tried the following.

Restarting the vpxa service

Restarting Mgmt services (/etc/init.d/hostd restart and services.sh restart)

Restarting the Veeam Backup and Replication console and backup proxies.

examining /var/log/hostd.log for any locked .vmdk files while running disk consolidation

I haven't been able to reboot the ESXi host or machine it's self, nor am I sure that will do much.

Does anyone have any suggestions?

Many Thanks!

Erik

[daphnissov]

Find out who holds the lock and remove it according to this KB.

[erikberndt]

Running

vmfsfilelockinfo -p /vmfs/volumes/574f021

9-de7758c4-5ce9-f48e38c19d8e/[VM]/[VM-NAME]-000021-delta.vmdk

I found 21 delta vmdk, some locked in read-only mode, others locked in exclusive mode. The offending MAC address belongs to the Virtual NIC attached to your Veeam backup server. There's a replication job running currently, but once that's done I'm going to try and reboot the Veeam server. Barring that, according to the KB, my options are to migrate the VM to the offending ESXI host (which is the one holding the lock) or restart the host itself. Is there really no other way to remove the file lock without rebooting the entire host?

[a_p_]

... but once that's done I'm going to try and reboot the Veeam server.

Also check whether the Veeam Backup server, or one of its proxies (if used) still have hot-added virtual disks/snapshots from the VM in question attached. If so, remove them (do not select the delete form disk option!)

André

[erikberndt]

Thanks André! That was it. Removing the stuck hot-added disk from one of the proxies removed the lock and let me consolidate :smileyhappy:

[john schuffert]

I had a system that I couldn't consolidate due to a locked file. We use Acronis and after verifying the Acronis Appliances did not have a lock on the system and after powering off and migrating, etc., etc. I removed it from inventory and re-registered the .vmx file and I was able to consolidate the disk. That was after running command line commands to check for locked files of which it said there weren't any. Last resort is to remove from inventory and register the .vmx file that will get you back in operation.

[Macatalina]

Where in Veeam can you check if the proxy is holding a snapshot?

[IRIX201110141]

Not in veeam.... just check the VM Settings within vSphere Client for your Virtual Proxy VM.

Regards,
Joerg

[PhilL123456]

hi 

How do you remove the stuck hot-added disk?   

thanks

Phil

[IRIX201110141]

Just open the VM settings of the given Veeam Proxy, expand the list of "harddisk" and search for the one which was sticked on the proxy.  If youre sure that the Backup isnt running any more remove the disk from the VM.   Dont tick the check box "delete from disk".

After that the consolidation function should be able to merge/commit the snapshot data back into the original vmdk.

Regards,
Joerg

 

[Macatalina]

Where in Veeam can you check if a proxy is holding a snapshot or disk

 

 

[PhilL123456]

I have similar issue with veeam.  I do see the proxy has additional hot-added virtual disk.  What is the proper way to remove it and what do you mean when you said (do not select the delete form disk option!)?

[ramin_ismail]

I accomplish this using the following.

I did shutdown to Veeam backup server. Then made a consolidation. It worked.

[bretua]

the easiest solution of them all. Worked like a charm. Thank you very much.

[theryangoodman]

Can't believe I didn't even try this first time round, I was off doing the more advanced fixes!

For anyone back at this post, try this step first!

 

Thanks a lot!

[IRIX201110141]

Thats a bad decision because there is a chance for more trouble. For sure the esxi will now able to consolidate the vDisk because you have shutdown the last VM which holds the lock on the vmdk because the proxy was not up and running any more and released the lock from the base disk.

This may ends up in a situation where you have 2 VMs referring to the same VMDK.

This problem are facing all backup solutions which use the hot add method aka a temporarily mount of a base vmdk on a 2nd. VM (proxy). I see it more often on undersized or very busy systems.

Speaking of Veeam you need to know that when Veeam BR Server is running as a VM the "proxy" role is part of the default installation.

Regards,
Joerg

[alt_n]

Snapshot can be created by third-party software during backup. Some snapshots did exist but you can't find it. One of the solution is manually creating one snapshot and then select Deleting All Snapshots.

By the way, we use Vinchin Backup & Recovery as our solution, and it also works well in our IT environment.