Script that looks at disk device usage (%USD in ESXtop)

[San-Infinity]

Hey guys,

Im trying to figure out how i can monitor the disk usage through PowerCLI but im getting stuck.

Currently a company im working for have installed SD-Cards as their boot device. They also have a scratch location on it. These SD-cards have started to degrade after years of usage, to the point the ESXi node stops working. We are working on replacing them.

The company has asked me to find ways to monitor whether an SD-Cards starts acting out, so we could preemptively put the host in maintenance and start swapping the SD-card out with another one.

Currently i have a script that is going through the VMKernel log and filters on "*failed H:0x7*"  Which should indicate that an iscsi device has failed working (we got those errors on host on which the SD-card was failing).

I wanted to add a script that looks at the usage of the SD card (called mpx.vmhba32:C0:T0:L0) because if the SD card was degraded, its usage would be 100% onder ESXITOP command (%USD would be 100).

But i sadly cant figure it out. I tried the get-esxtop command but i cant figure out if i can filter it on getting the disk device % usage like ESXTOP command (with the switch "u") does.

For those wondering, this is my current PowerCLI script (it still is a work in progress thing):

 

Connect-VIServer %vCentername%

$VMhosts = Get-VMHost -PipelineVariable esxihost

$value = ForEach ($VMhost in $VMhosts) {

$esxcli = get-vmhost $VMhost | Get-EsxCli

$esxcli.VMHost.Name
(Get-Log -VMHost (Get-VMHost $esxcli.VMHost.Name) vmkernel).Entries | Where {$_ -like “*failed H:0x7*“}
}
$attachment = "c:\temp\logs.txt"

$value | Out-File $attachment

if ($value -like "*failed H:0x7*")
{
Send-MailMessage -To “<recipient’s email address>” -From “<sender’s email address>” -Subject “xxx” -Body “xxx” -Attachments $attachment -Credential (Get-Credential) -SmtpServer “<smtp server>” -Port 587
}

else
{
Write-Host "its all good"
}

 

[LucD]

The Get-EsxTop cmdlet is a bit of hard nut to crack.
I did some attempts, see here 

Using the definition in Interpreting esxtop Statistics (section 4.2.1) for %USD, I came up with something like this.
Note that the loop sleeps for 5 seconds, because that is the esxtop interval.

Can you check if this indeed returns 100% for %USD for the disk involved?

$device = 'mpx.vmhba32:C0:T0:L0'

while($true){
  $result = Get-EsxTop -CounterName SCSIDevice | where{$_.DeviceName -eq $device}
  $actv = $result | Select-Object -ExpandProperty NumOfActiveCommands
  $qlen = $result | Select-Object -ExpandProperty QueueDepth
  $usd = $actv/$qlen
  Write-Host "$(Get-Date -Format hh:mm:ss.ffff ) %USD = $usd"
  Start-Sleep -Seconds 5
}

 

[San-Infinity]

Thanks man! To be honest, i didnt expected a reply that fast. I really appreciate this . VMware really has an awesome community.

 

Im going to test the script today  and see if it fits my case and ill update this post with the new script (for others to use if they want, i know ive used a bunch of scripts ive found on this forum!). 

EDIT:

Hands down! I bow to thee sir. The script does exactly what i wanted/asked for. I still need to delve into the "why" but i have found your blog on it and definitely am going to take a read. You just got yourself a new fan!