Disk Consolidation Stuck

[cjstout2050]

I have VM1 that is provisioned a 2TB thin disk. It is only roughly 1.2TB used inside Windows

Something happened on Tuesday night during our backup window, I believe the ESXi host crashed during the backup. The VMDK file ballooned up to 2TB, filling the datastore. I can see the error inside Veeam where it begins the disk consolidation process due to a stuck snapshot.

When I woke up Wednesday morning, I tried a disk consolidation with about 50GB left on the Datastore. This disk consolidation failed, filled up the datastore even more, and my VM would not run.

Eventually we got a workaround by spinning up an NFS datastore with a NAS on site, migrating the one other server VM2 over to it and clearing enough space(roughly 70GB) to at least let the VMs run.

My plan was on Friday evening to take a full backup, and then migrate my VM1 over to it, run the disk consolidation, then migrate it back. Unfortunately I get an error when doing this "Error caused by file Error caused by file /vmfs/volumes/5b638b75-f6f7e2e8-595a-98f2b33e1eec/*************1_1.VMDK"

So now my next plan is to just run a full backup with Veeam, and then restore it to my NFS datastore, make sure it's good, then delete the old and migrate it over, which is going to take about 30 hours to do. 

My question is I have read a bit into the disk consolidation process, I only have 2 VMDK files. One for my C drive, one for my large 2TB D Drive, and then a 1.7GB one which I believe is my stale snapshot/disk that needs consolidation. Why does my Datastore need so much free space to consolidate? Would powering off the VM and running the disk consolidation be successful? Last time I ran the consolidation it just filled up the datastore more.

[a_p_]

Without seeing the real sizes, it's not easy to help.

Anyway, a thin provisioned virtual disk can grow up to its provisioned size (2 TB in your case).
As an example: If the base/flat file's current size is 1.5TB, and you have a snapshot with 1.7TB, the base/flat file will grow to something between 1.7TB an 2.0TB, depending on what which data blocks have been modified.

With other words, the max. temporarily required free disk space is either flat size + snapshot size or provisioned size - flat size, whichever is less.

If you are willing to post a complete file listing (the output of ls -lisa from the command line) of the VM's files, I may be able to be more precise. You can of course grey out the VMs name (but not the numbers) if you want.

André

[cjstout2050]

This makes sense to me.

I did end up fixing the issue by running a backup with Veeam. I guess when Veeam did the snapshot and then removed it, it fixed the disk consolidation part.

Then I just ran vmfkstools -K disk.vdmk as described here https://kb.vmware.com/s/article/2136514

 

[AnaghB]

Hello  ,

The VM is using 1.2 TB inside the Guest OS but does the Esxi sees the same utilization on the disk? If Windows shows the usage as 1.2TB but Esxi is showing the usage as complete 2 TB then the blocks that are deleted by windows OS are not reclaimed on Esxi level. 

The solution is:

1. Run sdelete command on disk from OS.

2. Power OFF the VM and run the vmkfstools -k command on the vmdk from Esxi Putty session.

Post that check the usage and Esxi should show the usage as 1.2 TB. Please upgrade Tools to latest version and enable the parameter on Esxi configuration named OS Unmap to True so that if the OS zeros the deleted blocks the Esxi can read it.