IO utilizations (Read, Write IOPS and throughput, transfer/block size) For ALL VM in VC

[scorpion20]

Hi Team,

Is there a way to get IO utilizations (Read, Write IOPS and throughput, transfer/block size) for all the VM's in the virtual center, and output file generated in excel or CSV for the output? We need this data to examine the current infra load and then prepare the new backup solution based on this report. Any help is appreciated on this.

Regards,Nitin!

[LucD]

Besides the block size, you can get those metrics with the Get-Stat cmdlet.
Use the Get-StatType cmdlet to check which metrics are available in your environment.

[scorpion20]

Hi LucD, Thanks for your response.

 

I checked the below command Get-StatType -Entity $vm -Realtime and it showed way many parameters. Will the below script give the IOPS for the VM for the entire infra and put the output in the excel file? What about the throughput and the transfer block size, how can those be obtained?

Attachment(s)

IOPS Script.txt   680 B 1 version

[LucD]

The blocksize question I already answered.
For the throughput, you might want to look at the virtualdisk.throughput.usage.average metric.
See also Virtual Disk Counters for more available metrics.
To calculate the IOPS have a look at my Get the maximum IOPS post.

 

[scorpion20]

Hi LucD,

Thanks for the response. I tried the below script but this is for 1 vm. Can we get this run on the VC level/ Host level. As this it for all the VM could choke the network?

# Connect to vCenter Server
Connect-VIServer vm000002915.boigroup.net

$vmNames = "vm000004854"

# Specify the time range for which you want to get the IOPS data (in minutes)
$startTime = (Get-Date).AddMinutes(-60) # Adjust the time range as needed
$endTime = Get-Date

# Create an array to store the combined report data
$combinedReportData = @()

foreach ($vmName in $vmNames) {
# Get IOPS statistics for the specified VM within the specified time range
$iopsStats = Get-Stat -Entity $vmName -Stat disk.numberRead.summation, disk.numberWrite.summation -Start $startTime -Finish $endTime

# Check if statistics are available for the current VM
if ($iopsStats -ne $null) {
# Create an array to store the report data for the current VM
$reportData = @()

foreach ($stat in $iopsStats) {
$time = $stat.Timestamp
$readIOPS = $stat.ExtensionData.Value[0].Value
$writeIOPS = $stat.ExtensionData.Value[1].Value

# Create a custom object to store the data
$dataObject = [PSCustomObject]@{
VMName = $vmName
Time = $time
ReadIOPS = $readIOPS
WriteIOPS = $writeIOPS
}

# Add the custom object to the array
$reportData += $dataObject
}

# Add the VM-specific report data to the combined report data
$combinedReportData += $reportData
}
else {
Write-Host "No data available for VM: $vmName within the specified time range."
}
}

# Export the combined report data to a CSV file
$combinedReportData | Export-Csv -Path "Combined_VM_IOPS_Report.csv" -NoTypeInformation

# Display a message
Write-Host "Combined VM IOPS Report for specified VMs has been generated and saved as Combined_VM_IOPS_Report.csv"

 

[LucD]

You can change your foreach loop.

For all VMs in the vCenter

foreach ($vmName in Get-VM) {

For all VMs on a specific ESXi node

foreach ($vmName in (Get-VMHost -Name MyEsx | Get-VM)) {

This shouldn't normally have a huge impact on your network, since you are doing this VM per VM.
A faster way would be to do just 1 Get-Stat call for all VMs.
Then use the Group-Object cmdlet to divide the results per VM.
See for example my Limit cmdlet calls in a loop post.

[scorpion20]

Thank you very much for your input LucD.

I ran the below script and it completed, created the csv file but the file is empty. Anything wrong I am doing here?

(I am new to scripting so kindly ignore any silly questions)

# Connect to vCenter Server
Connect-VIServer VC-NAME

foreach ($vmName in (Get-VMHost -Name MyEsx | Get-VM)) {

# Specify the time range for which you want to get the IOPS data (in minutes)
$startTime = (Get-Date).AddMinutes(-60) # Adjust the time range as needed
$endTime = Get-Date

# Create an array to store the combined report data
$combinedReportData = @()

foreach ($vmName in $vmNames) {
# Get IOPS statistics for the specified VM within the specified time range
$iopsStats = Get-Stat -Entity $vmName -Stat disk.numberRead.summation, disk.numberWrite.summation, virtualdisk.throughput.usage.average -Start $startTime -Finish $endTime

# Check if statistics are available for the current VM
if ($iopsStats -ne $null) {
# Create an array to store the report data for the current VM
$reportData = @()

foreach ($stat in $iopsStats) {
$time = $stat.Timestamp
$readIOPS = $stat.ExtensionData.Value[0].Value
$writeIOPS = $stat.ExtensionData.Value[1].Value

# Create a custom object to store the data
$dataObject = [PSCustomObject]@{
VMName = $vmName
Time = $time
ReadIOPS = $readIOPS
WriteIOPS = $writeIOPS
}

# Add the custom object to the array
$reportData += $dataObject
}

# Add the VM-specific report data to the combined report data
$combinedReportData += $reportData
}
else {
Write-Host "No data available for VM: $vmName within the specified time range."
}
}}

# Export the combined report data to a CSV file
$combinedReportData | Export-Csv -Path "Combined_VM_IOPS_Report.csv" -NoTypeInformation

# Display a message
Write-Host "Combined VM IOPS Report for specified VMs has been generated and saved as Combined_VM_IOPS_Report.csv"

[LucD]

Did you replace the name 'MyEsx' with the name of your ESXi node?

[scorpion20]

Hi,

Yes replaced the 'MyEsx' with the name of ESXi node and then only it ran. But the csv file created is empty.

Thanks.

[LucD]

These metrics require statistic levels 3 and 4.
If you don't have that set, there will be no values for those metrics.
To understand what the statistics levels mean have a look at my PowerCLI & vSphere statistics – Part 1 – The basics post.

You can find which statistics level is set for each Interval with

 

 

 

$si = Get-View ServiceInstance 
$perfMgr = Get-View $si.Content.PerfManager 
$perfMgr.HistoricalInterval 

 

With the Realtime interval, you can get these metrics, independent from the statistics level.
But the Realtime interval only has metrics for approximately the last hour.

You could do something like this

 

$sStat = @{
  RealTime = $true
  Entity = Get-VMHost -Name MyEsx | Get-VM
  Stat = 'disk.numberRead.summation', 'disk.numberWrite.summation'
  Start = (Get-Date).AddMinutes(-60)
  Finish = Get-Date
  ErrorAction = 'SilentlyContinue'
}
Get-Stat  |
Sort-Object -Property Timestamp |
Group-Object -Property {$_.Entity.Name} -PipelineVariable VM |
ForEach-Object -Process {
  $_.Group | Group-Object -Property Timestamp -PipelineVariable Time |
  ForEach-Object -Process {
    [PSCustomObject]@{
      VMName = $VM.Name
      Time = $Time.Name
      NumberRead = $time.Group.Where{ $_.MetricId -eq 'disk.numberRead.summation'}.Value
      NumberWrite = $time.Group.Where{ $_.MetricId -eq 'disk.numberWrite.summation' }.Value
    }
  }
}

 

 

Note1: I used 'splatting' for the Get-Stat cmdlet which makes reading the cmdlet and parameters a lot easier.
Note2: these metrics do not return the IOPS value, you will have to calculate them with

    Disk reads/sec + disk writes/sec = IOPS

and don't forget that the interval is 20 seconds, so you will have to divide the value by 20 to get IOPS (the S in IOPS stands for 'second')

[BarryGrowler]

Consider using Get-VMHostHardwareIOP to retrieve IOPS metrics for all VMs across vCenter without extensive stats setup. Iterate through hosts, include the hostname, then export the compiled VM IOPS data to CSV. This lightweight approach avoids environment impact while providing the needed IOPS report.

[LucD]

I might be missing something, but Get-VMHostHardwareIOP isn't a PowerCLI cmdlet afaik.

[BarryGrowler]

You're correct, LucD. There is no such cmdlet called "Get-VMHostHardwareIOP" in the VMware PowerCLI module,sorry. The recommended approach to retrieve IOPS metrics for all VMs across a vCenter environment would be to use the "Get-Stat" cmdlet along with the appropriate performance counter IDs.