Automation

 View Only
Expand all | Collapse all

Way to report on Orphaned .VMDK files

sp93

sp93Jul 29, 2009 11:53 AM

  • 1.  Way to report on Orphaned .VMDK files

    Posted Jul 27, 2009 07:11 PM

    I have looked in multiple locations, however aren't finding what I need. Many scripts will only report on clusters, or will also register the found VMDK files, which is what I don't want.

    I would love (as I'm sure others would) to have a report in a .csv format that will scan my ESX 3.5 update 4 servers in my VC 2.5 update 4, and report any .VMDK files that aren't registered in VC.

    We have iSCSI, storage as well as local storage being used on each server.

    I have searched high and low for something, yet I can't seem to find anything that works.

    thanks in advance.



  • 2.  RE: Way to report on Orphaned .VMDK files

    Posted Jul 28, 2009 07:56 AM

    I started from the script in and made some, in my opinion, improvements.

    *) used the disk layout from the VirtualMachine objects since this contains all the .VMDK files

    *) included the .VMDK files from templates

    *) optimised the search method by only looking for .VMDK files

    *) changed the logic to determine if the .VMDK file is orphaned or not

    *) exported the results to a CSV file

    #
    # Purpose : List all orphaned vmdk on all datastores in all VC's
    # Version: 1.1
    # Author  : HJA van Bokhoven
    # Modifications: LucD
    
    $arrayVC = <VC1-name>, <VC2-name>
    $report = @()
    
    foreach ($strVC in $arrayVC)
    {
    	Connect-VIServer $strVC 
    	$arrUsedDisks = Get-View -ViewType VirtualMachine | % {$_.Layout} | % {$_.Disk} | % {$_.DiskFile}
    	$arrDS = Get-Datastore | Sort-Object -property Name
    	foreach ($strDatastore in $arrDS)
    	{
    		Write-Host $strDatastore.Name
    		$ds = Get-Datastore -Name $strDatastore.Name | % {Get-View $_.Id}
    		$fileQueryFlags = New-Object VMware.Vim.FileQueryFlags
    		$fileQueryFlags.FileSize = $true
    		$fileQueryFlags.FileType = $true
    		$fileQueryFlags.Modification = $true
    		$searchSpec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
    		$searchSpec.details = $fileQueryFlags
    		$searchSpec.matchPattern = "*.vmdk"
    		$searchSpec.sortFoldersFirst = $true
    		$dsBrowser = Get-View $ds.browser
    		$rootPath = ""
    		$searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec)
    
    		foreach ($folder in $searchResult)
    		{
    			foreach ($fileResult in $folder.File)
    			{
    				if ($fileResult.Path)
    				{
    					if (-not ($arrUsedDisks -contains ($folder.FolderPath + $fileResult.Path))){
    						$row = "" | Select DS, Path, File, Size, ModDate
    						$row.DS = $strDatastore.Name
    						$row.Path = $folder.FolderPath
    						$row.File = $fileResult.Path
    						$row.Size = $fileResult.FileSize
    						$row.ModDate = $fileResult.Modification
    						$report += $row
    					} 
    				}
    			}
    		} 
    	} 
    	# Disconnect session from VC
    	disconnect-viserver -confirm:$false
    }
    
    $report | Export-Csv "C:\VMDK-orphaned.csv" -noTypeInformation
    

    Please check if the files that are found by this script are indeed orphaned before deleting them !



  • 3.  RE: Way to report on Orphaned .VMDK files

    Posted Jul 28, 2009 11:59 AM

    Thanks so much, the report seems correct, however my only issue is that the ESX server name isn't listed. So for the orphaned .vmdk files that sit on local storage, it's hard to locate what server they are actually on.

    Is there anyway that the ESX server could be one of the columns?

    thanks so much



  • 4.  RE: Way to report on Orphaned .VMDK files

    Posted Jul 28, 2009 12:54 PM

    Sure, this should do the trick.

    Note that if a datastore is shared between multiple ESX hosts you will see the name of the first ESX host in the CSV file.

    #
    # Purpose : List all orphaned vmdk on all datastores in all VC's
    # Version: 1.1
    # Author  : HJA van Bokhoven
    # Modifications: LucD
    
    $arrayVC = <VC1>,<VC2>
    $report = @()
    
    foreach ($strVC in $arrayVC)
    {
    	Connect-VIServer $strVC 
    	$arrUsedDisks = Get-View -ViewType VirtualMachine | % {$_.Layout} | % {$_.Disk} | % {$_.DiskFile}
    	$arrDS = Get-Datastore | Sort-Object -property Name
    	foreach ($strDatastore in $arrDS)
    	{
    		Write-Host $strDatastore.Name
    		$ds = Get-Datastore -Name $strDatastore.Name | % {Get-View $_.Id}
    		$fileQueryFlags = New-Object VMware.Vim.FileQueryFlags
    		$fileQueryFlags.FileSize = $true
    		$fileQueryFlags.FileType = $true
    		$fileQueryFlags.Modification = $true
    		$searchSpec = New-Object VMware.Vim.HostDatastoreBrowserSearchSpec
    		$searchSpec.details = $fileQueryFlags
    		$searchSpec.matchPattern = "*.vmdk"
    		$searchSpec.sortFoldersFirst = $true
    		$dsBrowser = Get-View $ds.browser
    		$rootPath = ""
    		$searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec)
    
    		foreach ($folder in $searchResult)
    		{
    			foreach ($fileResult in $folder.File)
    			{
    				if ($fileResult.Path)
    				{
    					if (-not ($arrUsedDisks -contains ($folder.FolderPath + $fileResult.Path))){
    						$row = "" | Select DS, Path, File, Size, ModDate, Host
    						$row.DS = $strDatastore.Name
    						$row.Path = $folder.FolderPath
    						$row.File = $fileResult.Path
    						$row.Size = $fileResult.FileSize
    						$row.ModDate = $fileResult.Modification
    						$row.Host = (Get-View $ds.Host[0].Key).Name
    						$report += $row
    					} 
    				}
    			}
    		} 
    	} 
    	# Disconnect session from VC
    	disconnect-viserver -confirm:$false
    }
    
    $report | Export-Csv "C:\VMDK-orphaned.csv" -noTypeInformation
    



  • 5.  RE: Way to report on Orphaned .VMDK files

    Posted Jul 29, 2009 11:53 AM

    Awesome, thanks so much



  • 6.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 03, 2009 03:01 PM

    Any ideas why I get this error?

    Exception calling "SearchDatastoreSubFolders" with "2" argument(s): "Invalid datastore path ''."

    At 30 char:55

    + $searchResult = $dsBrowser.SearchDatastoreSubFolders &lt;&lt;&lt;&lt; ($rootPath, $searchSpec)

    + CategoryInfo : NotSpecified: (:smileyhappy: [], MethodInvocationException

    + FullyQualifiedErrorId : DotNetMethodException



  • 7.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 03, 2009 03:58 PM

    Did you copy the script from your browser or did you use the attached file ?

    The $rootPath variable contains some square brackets and the forum SW has problems with displaying those correctly.

    Try the attached script.



  • 8.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 04, 2009 01:30 AM

    LucD,

    Unfortunately that didn't work. Same error.

    ta.



  • 9.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 04, 2009 01:37 AM

    HAHA yes it did fix it... stupid me, updated the script in a different folder to where my PS window was running it from.

    thanks LucD.



  • 10.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 04, 2009 03:47 PM

    Hi, Thanks for the Script. Unfortunately I cannot get it to work. I downloaded the attached script and all I did was modify the arrayvc line to look like this: $arrayVC = "vcentersrvr"

    I keep getting this error:

    Exception calling "SearchDatastoreSubFolders" with "2" argument(s): "Not initia

    lized: boolean fileOwner"

    At C:\Get-orphaned-VMDK.ps1:29 char:55

    + $searchResult = $dsBrowser.SearchDatastoreSubFolders &lt;&lt;&lt;&lt; ($rootPath,

    $searchSpec)

    + CategoryInfo : NotSpecified: (:smileyhappy: [], MethodInvocationException

    + FullyQualifiedErrorId : DotNetMethodException

    Any thoughts? I am using vSphere 4. Thanks very much.

    Mike



  • 11.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 04, 2009 08:40 PM

    The $arrayVC variable should be an array.

    And if you want to create an array with only 1 element you should do

    $arrayVC = @("vcentersrvr")
    

    Try the script with that adaptation.



  • 12.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 04, 2009 09:15 PM

    hmmm ...... I made the modification you suggested and I still get the exact same errors. I see that the script conncects to my VC and then begins pulling all the datastores, but the error is being displayed everytime DS is displayed on the screen. Any further thoughts? Thanks for the help. I'm very rusty in powershell.

    Mike



  • 13.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 04, 2009 09:29 PM

    You're right, should have read your error message more accurately.

    It seems that the FileQueryFlags object in vSphere has a new property called fileOwner.

    This is a Boolean that needs to be set to $false or $true.

    So these lines

    ...
    		$fileQueryFlags = New-Object VMware.Vim.FileQueryFlags
    		$fileQueryFlags.FileSize = $true
    		$fileQueryFlags.FileType = $true
    		$fileQueryFlags.Modification = $true
    ...
    

    should be

    ...
    		$fileQueryFlags = New-Object VMware.Vim.FileQueryFlags
                   $fileQueryFlags.fileOwner = $true                   # Only for vSphere environments !
    		$fileQueryFlags.FileSize = $true
    		$fileQueryFlags.FileType = $true
    		$fileQueryFlags.Modification = $true
    ...
    

    when you run the script in a vSphere environment.



  • 14.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 04, 2009 10:10 PM

    Sadly no luck. I copied your new lines and pasted them over the existing lines .... same error:

    CX400_500GB_DS4

    Exception calling "SearchDatastoreSubFolders" with "2" argument(s): "Not initia

    lized: boolean fileOwner"

    At C:\Get-orphaned-VMDK.ps1:30 char:55

    + $searchResult = $dsBrowser.SearchDatastoreSubFolders( <<<< $rootPath,

    $searchSpec)

    Thanks very much for the help!



  • 15.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 05, 2009 07:56 PM

    There's something fishy going on :smileysad:

    I could reproduce the problem with PowerCLI 4 in a vSphere environment.

    @VMware Could this be a bug ?



  • 16.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 05, 2009 08:57 PM

    Carter confirms this is a bug, although it's not clear if it's a PowerCLI or a API bug.

    The bypass is to leave out the details property completely.

    In this script the details of the files (.vmdk) that are returned are not used in the script.

    Attached a "vSphere" version of the script (with the "details" property left out).



  • 17.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 05, 2009 10:00 PM

    The script runs without any runtime errors now and the output of the CSV file looks awesome. I could definitely see a use to schedule this once a Month as part of my health scripts, but sadly, there is still a couple of problems. All of the vmdk files show 0 in size and a ModDate of 1/1/0001 12:00:00 AM.

    I appreciate you help very much and I hate to bother you more, but is there any chance you can dig a little deeper to resolve that issue?

    Thanks very much.



  • 18.  RE: Way to report on Orphaned .VMDK files

    Broadcom Employee
    Posted Aug 06, 2009 09:02 AM

    Hi,

    All of the vmdk files show 0 in size and a ModDate of 1/1/0001 12:00:00 AM.

    This is caused by the the bypass of the details property completely.

    I've modified the script to workaround the bug LucD mentioned so I hope this will solve your task.

    Regards,

    Yasen



  • 19.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 06, 2009 10:45 AM

    Thanks Yasen.

    Where can we find more info on \[VMware.Vim.VIConvert\] ?



  • 20.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 06, 2009 01:50 PM

    Awesome. Thank you both very much! It seems to work perfectly now. This will be very handy indeed.



  • 21.  RE: Way to report on Orphaned .VMDK files

    Posted Nov 10, 2009 11:16 AM

    I'm also try to get more info about vmware.vim.viconvert but no luck. Any idea how to convert this script in vb.net using vmware.vim dll as I can't find any resource available? It seems vimapi_40 class from vmware.vim is different from vim25api class for vSphere 4.



  • 22.  RE: Way to report on Orphaned .VMDK files

    Posted Sep 29, 2009 07:47 PM

    I've tried to run this with teh updated code for vSphere, but I keep getting this error:

    Exception calling "SearchDatastoreSubFolders" with "2" argument(s): "Datastore '[SAN001 (LUN 804)]' is not accessible."

    At :line:38 char:55

    ++ $searchResult = $dsBrowser.SearchDatastoreSubFolders &lt;&lt;&lt;&lt; ($rootPath, $searchSpec)+

    This isnt the first lun it hits. I've tried excluding that LUN thinking there was an issue.. but it will just hang up on the next LUN. I checked that I can browse them just fine.



  • 23.  RE: Way to report on Orphaned .VMDK files

    Posted Nov 12, 2009 08:56 AM

    Is there a solution now for the "Not initialized: boolean fileOwner" Problem ?

    I have excact the same Problem a s mentioned above, even when i set the $searchSpec.Details.FileOwner =$true explicit.

    I'm using PowerCLI 4.0 build 162509 on a VSphere environment.

    Greetings



  • 24.  RE: Way to report on Orphaned .VMDK files

    Posted Nov 12, 2009 09:02 AM

    Yes, see Yasen's reply in



  • 25.  RE: Way to report on Orphaned .VMDK files

    Posted Jul 19, 2010 07:10 PM

    Thanks a lot Lucd for this great and very useful script, and Yasen for the work around for vSphere 4 fileOwner bug.

    I'm new to powercli and vsphere scripting and I have one question, How can I suppress the ctk.vmdk files from the results, since they aren't actually orphaned vmdks but "change tracking" vmdk files?

    Or even better, is it possible to just show the ctk.vmdk file if it is associated with a vmdk file which in turn is orphaned? In the mean time I will keep looking into this and will post any findings I might have...

    Thanks in advance,

    Kind regards from Brazil,

    Paulo



  • 26.  RE: Way to report on Orphaned .VMDK files

    Posted Jul 19, 2010 07:39 PM

    You could try changing the test

    if (-not ($arrUsedDisks -contains ($folder.FolderPath + $fileResult.Path))){
    

    into

    if (-not ($arrUsedDisks -contains ($folder.FolderPath + $fileResult.Path) -and $fileResult.Path -notmatch "ctk.vmdk ")){
    

    ____________

    Blog: LucD notes

    Twitter: lucd22



  • 27.  RE: Way to report on Orphaned .VMDK files

    Posted Jul 19, 2010 08:53 PM

    It worked!

    Just had to change the parenthesis to

    if (-not ($arrUsedDisks -contains ($folder.FolderPath + $fileResult.Path)) -and $fileResult.Path -notmatch "ctk.vmdk"){
    

    Thanks a lot!

    PS. An observation: Yasen workaround also suppressed flat.vmdk from the results (which appeared when the "details" property were left out) maybe there is a way of using "VMware.Vim.VIConvert" as a way of tying ctk with its respective vmdk and that way if there is an orphaned ctk file it would show up on the results. Not that it matters since CTK files ain't that big, but I thought it was worth mentioning.



  • 28.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 09, 2009 03:51 AM

    I'm running the script against some of my ESX servers. It processed fine for a number of servers, but it's hanging (not crashing, not failing - just continuing to process w/o any CPU activity) at the following line for a long time - probably 10 minutes at least.

    $searchResult = $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec)

    The ESX servers are all on the same network and I'm running the script on the VirtualCenter Server.

    Is it normal for "SearchDatastoreSubFolders" method to take such a long time? I did browse the datastores through Infrastructure Client and browsing some of the datastores on the same ESX servers took a VERY long time too. Any idea what could be causing it to be so slow?

    Thanks,

    Chris



  • 29.  RE: Way to report on Orphaned .VMDK files

    Posted Aug 09, 2009 05:21 AM

    Never mind.

    After the datastore browser finished searching for files I found that there were a couple thousand old log files in several of the folders in the datastore. I'm guessing this is why it's slow, but it would be nice if I could speed up the search.

    Great Script!

    Thanks,

    Chris



  • 30.  RE: Way to report on Orphaned .VMDK files

    Posted Apr 02, 2011 04:51 AM

    Hi Luc,

    In my environment there is a known problem that a couple of datastore had VMFS crash and none of the esx box could read the contents of datastore. I'm running this script Get-orphaned-vmdk and when this script tries to retrive the contents of the corrupted datastore in the line $dsBrowser.SearchDatastoreSubFolders($rootPath, $searchSpec) the script breaks and halts. Could you please help to skip the datastore where it can't get the contents and continue with remaining datastores?



  • 31.  RE: Way to report on Orphaned .VMDK files

    Posted Sep 24, 2009 06:59 PM

    I just wanted to chime in as well and say that this is an amazingly helpful script! It has helped me clear out much space on my datastores. I tore it down and wrote about a half dozen scripts based on it to report on several different types of file-related issues. (Log bloat, random .iso's, etc)

    LucD thanks for pointing out those fixes! I recently upgraded to vSphere and was worried about stuff like that breaking.