Zombie VMDK issues.

[Kylel]

Hi All,

I'm looking for a way to control or monitor the amount of zombie VMDKs that show up on our datastores. Currently, I manually check for them using RVTools about once a month and usually find about 30. It's a bit of a time consuming process double or triple checking that each one is actually unused then renaming (to be safe) then deleting from the datastore.

I'm assuming they exists due to some sort of bug in either svMotion (SRDS is enabled) or Veeam B&R which is used nightly on about 75VMs using hot add.

Is there a better way to monitor and manage zombie VMDKs? Or better yet, a way to prevent them from being created?

Kyle.

[aaronwsmith]

I assume you mean orphaned VMDKs.

They happen when someone removes, but does not delete, a virtual disk from a VM.  By default when you remove a virtual disk from a VM, it simply unregisters it, but leaves the VMDK on the datastore.  You have to explicitly select to delete from disk to erase them from the datastore.

PowerCLI will enable you to automatically scan and report on orphaned VMDKs.  There are already solutions out there for this.  Example:

Discovering orphaned vmdk files in vSphere « J. Gregs Brain Corral (this script is a little outdated, but still works if you remove the line of code on line 17 "[Reflection.Assembly]::LoadWithPartialName("VMware.Vim")")

[Kylel]

Thanks Aaron,

You're right, the correct terminology is probably orphaned VMDK, I was just following what RVTools calls them.

The strange thing is, no one is modifying these VMs. That's why I'm thinking Veeam is somehow causing it when hot adding the disks for backup. Eg, what would happen if SDRS moves a VM while Veeam has the disk mounted via hot add?

I'll take a look into using the script to email reports each day - I might be able to just clean the files up as they appear instead of as a bulk job.

Kyle.

[aaronwsmith]

Interesting.  Sorry I overlooked your comments on Veeam. and SDRS.

Next time you find orphaned VMDKs, can you share info on their file names and sizes?  The file names are different for snapshot VMDK files (which have "delta" in their name) vs. normal VMDK files.  Knowing this could help logically narrow down the source of the problem.

[continuum]

Be careful when using the results of RV-tools.

I found that the number of false positive Zombies detected by RV-tools is so high that I read the results as "the VMs listed in the Zombie column may eventually be Zombies"

I dont think that there is any tool out there that can detect Zombies with the probabilty you would want - to be able  to use the results for an automatic action - like delete the directory where the eventual Zombie lives in

[Kylel]

I've setup that script to send reports and it seems to be working. This will at least let me know when a zombie appears.

The file names and size are that of the original base disk - .vmdk and -flat.vmdk with the size depending on the VM, some file or db servers might have a 500GB disk. Occasionally I'll see a snapshot in there too 0000001.vmdk and 0000001-delta.vmdk.

RVTools has actually been pretty accurate so far but I am definitely not taking it as the final answer. I usually check the VM config to see which files the hard drives are linked to and go from there.

What do you think about the amount of these zombies that I'm getting? The clean up that I performed yesterday saved about 1TB (30ish vmdks) across the datastores and it's only been about 2 months since I last checked...

Kyle.