powercli script to capture cpu & mem usage stats of Individual VM

[vmhyperv]

Hi ,

Is it possible to find out or   to capture cpu & mem usage stats of individual VMs  for last 5 Days thru PowerCli and Export to CSV report

Thanks

vmguy

[LucD]

Have a look at Get VMs CPU and Memory Daily Usage.

You can easily change from 1 day to 5 days by doing

$start = (get-date).AddDays(-5)

[vmhyperv]

Hi LuCD,

       Thanks  for prompt help but i  want to pull out for only 1 VM  like VMSTPA3INDWH07  in CSV format then what will be the script

Thanks

vmguy

[vmhyperv]

LucD,

  its throwing error.Pls find the attached script.

[vSphere PowerCLI] E:\> .\vm-memory.ps1
The term ' ' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
again.
At E:\vm-memory.ps1:6 char:58
+ Get-Stat -Entity ($vms) -start $start -stat $metrics | `  <<<<  Group-Object -Property EntityId | %{
    + CategoryInfo          : ObjectNotFound: ( :String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

[LucD]

There was apparently a problem with your copy/paste. Some of the <CR><LF> got lost.

I attached a corrected version.

[vmhyperv]

Thanks LucD,

     But Its shows error

[vSphere PowerCLI] E:\> .\vm-memory-usage.ps1
Get-Stat : 7/26/2011 1:58:27 PM    Get-Stat        The metric counter "mem.active.average" doesn't exist for entity "USTPA3INDWH07".
At E:\vm-memory-usage.ps1:6 char:9
+ Get-Stat <<<<  -Entity ($vms) -start $start -stat $metrics | `
    + CategoryInfo          : ResourceUnavailable: (mem.active.average:String) [Get-Stat], VimException
    + FullyQualifiedErrorId : Client20_RuntimeDataServiceImpl_CheckUserMetrics_MetricDoesntExist,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetViStats

WARNING: 'Entity' property is obsolete. Use 'EntityId' instead.

PFA

[LucD]

Is that a new VM that hasn't been running long ?

Or has it been powered off for a longer period ?

[AviShimon]

Hi LucD,

i'm having a similar problem which following this discussion, i still do not get it to work.

when i run get-stattype for VM's ,I get the list of available metrics, but without "mem.active.average".

I have raised statistics level to 2 and still No 'mem.active..' metric.

i can get this for a specific VM using VC performance charts but it is not very usefull.

please advise.

Thanks, Avi

[LucD]

Which intervals did you specify ?

Try like this and see if it returns the metric(s) for any of the intervals

Get-StatType -Entity $vm -Interval "Past Day" | where {$_ -match "mem.active"}

Get-StatType -Entity $vm -Interval "Past Week" | where {$_ -match "mem.active"}

Get-StatType -Entity $vm -Interval "Past Month" | where {$_ -match "mem.active"}

Get-StatType -Entity $vm -Interval "Past Year" | where {$_ -match "mem.active"}

[AviShimon]

Hi,

thanks for your prompt reply.

I now get the metric for "Past Day"  interval .  i guess it need time to build for "Past Week" ( also in level 2).

Now this command Works :

PS C:\Windows\system32> Get-Stat -Entity (Get-VM my-vm) -Stat mem.active.average -IntervalMins 5 -MaxSamples 3 | where{$_.instance -eq ""}

MetricId                Timestamp                          Value Unit     Instance
--------                ---------                          ----- ----     --------
mem.active.average      24/01/2013 13:45:00                    0 KB              
mem.active.average      24/01/2013 13:40:00               748226 KB              
mem.active.average      24/01/2013 13:35:00               351286 KB    

But, when i"m trying to use the start/finish , i get the mistake,

$today = (Get-Date -Hour 10 -Minute 30 -Second 0)
$now = (Get-Date -Hour 13 -Minute 30 -Second 0)
Get-Stat -Entity (Get-VM my-vm) -Stat mem.active.average -Start $today -Finish $now | where{$_.Instance -eq ""}   

Get-Stat : 24/01/2013 13:53:14    Get-Stat        Object reference not set to an instance of an object.  
At line:3 char:9
+ Get-Stat <<<<  -Entity (Get-VM my-vm) -Stat mem.active.average -Start $today -Finish $now | where{$_.Instance -eq ""}
    + CategoryInfo          : NotSpecified: (:) [Get-Stat], VimException
    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetViStats    

please advise.

thanks, Avi 

[LucD]

Yes, it will take time to see the metric appear in all intervals.

Your lines of code seem to work for me. It looks as if the problem is that 'Get-VM my-vm' doesn't return anything.

[AviShimon]

Get-vm is the same on both lines , the only thing changed in the Time Limit/Frame.

if i want to get the data using max samples and interval  for a whole day , how would you suggest to do it , (or any other way )

Thanks,

[LucD]

Could it be that the VM was not running at the time specified in $start ?

Do you see statistics for that time in the vSphere client ?

[AviShimon]

VM is running, there are 2 adjacent powercli lines, one is working the other is Not, as I written , the only different is the Interval/Start/Finish.

Yes, i am monitoring VC performance couters of that VM at the same time.

Thanks, Avi

[LucD]

The only times I have seen that error with the Get-Stat cmdlet , is when there was no statistical data available at the time specified by the $Start variable.

Can you try changing the value in the $Start variable I see if that makes a difference ?

[AviShimon]

Hi LucD,

Problem Solved.

I guess it's kind of a bug in the powercli version that I used.

I have upgraded to verison 5.1 (latest) and things started to role.

Thanks for all your support.

i must say you're doing a Holy job in hte community (without offending anyone ...) :smileyhappy:

Avi

[LucD]

Thanks, glad your problem got solved

[Nsousa]

Originally I had issues with this script, but I turned on Statistics Level 2 in vCenter, and it began pulling all of the metrics I needed - or so I thought.

Right now it is still not populating the MinMem, AvgMem, and MaxMem columns. Any ideas?

[LucD]

I don't know which interval you are looking at, but if it goes back several days, you have to take the aggregation jobs that run on the vCenter database into account.

Have a look at my PowerCLI & vSphere statistics – Part 1 – The basics post where I try to explain the intervals and the aggregation jobs.

[Nsousa]

It is working with the statistics level set to 4 for the 5 minute / 30 minute intervals.

However, it is only pulling the specific days worth of data. Is there a way to have it pull all data since X days in the past?

[LucD]

Only if the historical interval you are looking at has the required statistical level and there has been sufficient time passed to have all the aggregation jobs propagate the required level to the interval.

Have a look at the diagram in my Part 1, it will indicate how long you will have to wait.

[Nsousa]

It is able to pull the data for 3/7, as well as 3/8 - but not in the same report. If I use AddDays(-1) it will pull the data from 3/8. If I change it to AddDays(-2), it will only pull the data from 3/7 but not 3/8.

Is that the intended behavior? Shouldn't it pull the metrics from both 3/7 and 3/8 if I select AddDays(-2)?

[LucD]

Are the interval values in the data returned for 3/7 and 3/8 the same ?

The data for 3/8 probably comes from Historical Interval 1, with a sampling interval of 5 minutes.

The data for 3/7 probably comes from Historical Interval 2 with a sampling interval of 30 minutes.

Is my assumption correct ?

[Nsousa]

No, it doesn't look like it.

This is the output with using $start = (get-date).AddDays(-2)

VmName	Timestamp	vCPU	MinCpu	AvgCpu	MaxCpu	MemAlloc	MinMem	AvgMem	MaxMem
Server01	3/10/2012 9:30	1	1.38	2.03	7.14	1024	6.63	9.8	35.89

This is the output with using $start = (get-date).AddDays(-1)

VmName	Timestamp	vCPU	MinCpu	AvgCpu	MaxCpu	MemAlloc	MinMem	AvgMem	MaxMem
Server01	3/11/2012 9:30	1	1.38	2.07	7.14	1024	6.9	10.65	35.89

$report = @()

$metrics = "cpu.usage.average","mem.usage.average"

$vms = Get-Vm Server01 | where {$_.PowerState -eq "PoweredOn"}

$start = (get-date).AddDays(-1)

Get-Stat -Entity ($vms) -start $start -stat $metrics | `

  Group-Object -Property EntityId | %{

    $row = ""| Select VmName, Timestamp, vCPU, MinCpu,AvgCpu,MaxCpu,MemAlloc,MinMem,AvgMem,MaxMem

    $row.VmName = $_.Group[0].Entity.Name

    $row.Timestamp = ($_.Group | Sort-Object -Property Timestamp)[0].Timestamp

    $row.vCPU = $_.Group[0].Entity.NumCpu

    $cpuStat = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} | Measure-Object -Property Value -Minimum -Maximum -Average

    $row.MinCpu = "{0:f2}" -f ($cpuStat.Minimum)

    $row.AvgCpu = "{0:f2}" -f ($cpuStat.Average)

    $row.MaxCpu = "{0:f2}" -f ($cpuStat.Maximum)

    $row.MemAlloc = $_.Group[0].Entity.MemoryMB

    $memStat = $_.Group | where {$_.MetricId -eq "mem.usage.average"} | Measure-Object -Property Value -Minimum -Maximum -Average

    $row.MinMem = "{0:f2}" -f ($memStat.Minimum)

    $row.AvgMem = "{0:f2}" -f ($memStat.Average)

    $row.MaxMem = "{0:f2}" -f ($memStat.Maximum)

    $report += $row

}

$report | Export-Csv "C:\VM-stats1.csv" -NoTypeInformation -UseCulture

[LucD]

Can you display the Interval property ?

Or 2 consecutive rows for each date, that way we can also see the interval.

[JayLB]

Played around with this one and have it returning one line per interval by changing EntityID to Timestamp.

$report = @()
$metrics = "cpu.usage.average","mem.usage.average"
$vms = Get-Vm | where {$_.PowerState -eq "PoweredOn"}
$start = (get-date).AddDays(-30)
Get-Stat -Entity ($vms) -start $start -stat $metrics | `
Group-Object -Property Timestamp | %{
    $row = ""| Select VmName, Timestamp, vCPU, MinCpu,AvgCpu,MaxCpu,MemAlloc,MinMem,AvgMem,MaxMem
    $row.VmName = $_.Group[0].Entity.Name
    $row.Timestamp = ($_.Group | Sort-Object -Property Timestamp)[0].Timestamp
    $row.vCPU = $_.Group[0].Entity.NumCpu
    $cpuStat = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} | Measure-Object -Property Value -Minimum -Maximum -Average
    $row.MinCpu = $cpuStat.Minimum
    $row.AvgCpu = $cpuStat.Average
    $row.MaxCpu = $cpuStat.Maximum
    $row.MemAlloc = $_.Group[0].Entity.MemoryMB
    $memStat = $_.Group | where {$_.MetricId -eq "mem.usage.average"} | Measure-Object -Property Value -Minimum -Maximum -Average
    $row.MinMem = $memStat.Minimum
    $row.AvgMem = $memStat.Average
    $row.MaxMem = $memStat.Maximum
}


 
$report | Export-Csv "D:\Client report\Performance\PerfData\VM Report.csv" -NoTypeInformation -UseCulture
 

[JayLB]

Noticed though all the max,min and avg return the same value per line when changing the Group-Object to Timestamp from EntityID.

Timestamp     vCPU     MinCpu             AvgCpu             MaxCpu             MemAlloc  MinMem     AvgMem               MaxMem
1/7/2013 12:00     2     1.580000043     1.580000043     1.580000043     4096     7.840000153     7.840000153     7.840000153
1/7/2013 10:00     2     1.580000043     1.580000043     1.580000043     4096     7.610000134     7.610000134     7.610000134
1/7/2013 8:00     2     1.549999952     1.549999952     1.549999952     4096     7.789999962     7.789999962     7.789999962
1/7/2013 6:00     2     1.620000005     1.620000005     1.620000005     4096     7.769999981     7.769999981     7.769999981
1/7/2013 4:00     2     1.669999957     1.669999957     1.669999957     4096     7.730000019     7.730000019     7.730000019
1/7/2013 2:00     2     1.480000019     1.480000019     1.480000019     4096     7.360000134     7.360000134     7.360000134
 

Any idea on eating my cake too.

[LucD]

Did you check how many metrics you have per group ?

If there is only 1 entry then the fact that you see the same values for Min/Max/Avg is normal, otherwise we need to look further into that.

[JayLB]

Got it.  And since I want point in time values to create a monthly graph from then I shouldn't measure the values.

Something like this:

 
$report = @()
$metrics = "cpu.usage.average","mem.usage.average","disk.maxTotalLatency.latest","net.usage.average"
$vms = Get-Vm | where {$_.PowerState -eq "PoweredOn"}
$start = (get-date).AddDays(-31)
Get-Stat -Entity ($vms) -start $start -stat $metrics -IntervalMins 120 | `
  Group-Object -Property {$_.Entity.Name, $_.Timestamp}| %{ 
    $row = ""| Select Timestamp, "Virtual Machine", "vCPU Count", "Avg Cpu Usage (%)", "vRam Provisioned", "Avg Memory Usage (%)", "Avg Disk Latency (ms)", "Avg Net Usage (kb)"
    $row."Virtual Machine" = $_.Group[0].Entity.Name
    $row.Timestamp = ($_.Group | Sort-Object -Property Timestamp)[0].Timestamp
    $row."vCPU Count" = $_.Group[0].Entity.NumCpu
    $cpuStat = $_.Group | where {$_.MetricId -eq "cpu.usage.average"}
    $row."Avg Cpu Usage (%)" = $cpuStat
    $row."vRam Provisioned" = $_.Group[0].Entity.MemoryMB
    $memStat = $_.Group | where {$_.MetricId -eq "mem.usage.average"}
    $row."Avg Memory Usage (%)" = $memStat
    $diskStat = $_.Group | where {$_.MetricId -eq "disk.maxTotalLatency.latest"}
    $row."Avg Disk Latency (ms)" = $diskStat
    $report += $row
    $netStat = $_.Group | where {$_.MetricId -eq "net.usage.average"}
    $row."Avg Net Usage (kb)" = $netStat
    $report += $row
  }
 

[LucD]

I would propose some minor changes

$report = @()
$metrics = "cpu.usage.average","mem.usage.average","disk.maxTotalLatency.latest","net.usage.average" 
$vms = Get-Vm | where {$_.PowerState -eq "PoweredOn"}

$start = (get-date).AddDays(-31)
Get-Stat -Entity ($vms) -start $start -stat $metrics -IntervalMins 120 |
  Group-Object -Property {$_.Entity.Name, $_.Timestamp}| %{ 
    $row = ""| Select Timestamp, "Virtual Machine", "vCPU Count", "Avg Cpu Usage (%)", "vRam Provisioned", "Avg Memory Usage (%)", "Avg Disk Latency (ms)", "Avg Net Usage (kb)"
    $row."Virtual Machine" = $_.Values[0]
    $row.Timestamp = $_.Values[1]
    $row."vCPU Count" = $_.Group[0].Entity.NumCpu
    $cpuStat = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} | Select -ExpandProperty Value
    $row."Avg Cpu Usage (%)" = $cpuStat
    $row."vRam Provisioned" = $_.Group[0].Entity.MemoryMB
    $memStat = $_.Group | where {$_.MetricId -eq "mem.usage.average"} | Select -ExpandProperty Value
    $row."Avg Memory Usage (%)" = $memStat
    $diskStat = $_.Group | where {$_.MetricId -eq "disk.maxTotalLatency.latest"} | Select -ExpandProperty Value
    $row."Avg Disk Latency (ms)" = $diskStat
    $netStat = $_.Group | where {$_.MetricId -eq "net.usage.average"} | Select -ExpandProperty Value
    $row."Avg Net Usage (kb)" = $netStat
    $report += $row
  }

The properties on which you group are available under the Values property in the objects returned by Group-Object.

You want the value and not the complete metric that is returned by Get-Stat. That's why the Select -ExpandProperty is there.

Removed the 2nd '$report += $row'.

Note that the PowerState test will only check if the VM is currently powered on. That doesn't guarantue that it was powered on at the start of the interval. So this test is not foolproof. You can examine the poweron poweroff events to test if a VM was powered on at the start of the interval, but the overhead and complexity of the script would probably be too much.

[JayLB]

Perfect!  Thanks LucD!

[JayLB]

After 12 hours I had to kill the process.  The Powershell process was running at 50% with 250mb consumed the whole time.  Modified the script with a foreach statement to break it up into VM size bites.  Takes about a minute per VM to run on this customer's vCenter.

Takes much longer to run but this script also reduced my csv file by 5 times of the size rather than collecting each metric by MetricID into a csv file.

170 VMs * 1 mins per = 2.8 hours to run

About 2.xMB of Mem per VM in Powershell.exe = 360-400MB Memory required for Powershell

# VM Performance metrics in a single line per interval

$report = @()

$metrics = "cpu.usage.average","mem.usage.average","disk.maxTotalLatency.latest","net.usage.average"

$vm = Get-Vm

$start = (get-date).AddDays(-31)

Foreach($vms in $vm) {

  Get-Stat -Entity ($vms) -start $start -stat $metrics -IntervalMins 120 |`

   Group-Object -Property {$_.Entity.Name, $_.Timestamp}| %{

    $row = ""| Select "DateTime", "Virtual Machine", "vCPU Count", "Avg Cpu Usage (%)", "vRAM Provisioned (GB)", "Avg Memory Usage (%)", "Avg Disk Latency (ms)", "Avg Net Usage (kb)"  

    $row."Virtual Machine" = $_.Group[0].Entity.Name

    $row."DateTime" = $_.Group[0].Timestamp

    $row."vCPU Count" = $_.Group[0].Entity.NumCpu

    $cpuStat = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} | Select -ExpandProperty Value

    $row."Avg Cpu Usage (%)" = $cpuStat

    $row."vRAM Provisioned (GB)" = $_.Group[0].Entity.MemoryGB

    $memStat = $_.Group | where {$_.MetricId -eq "mem.usage.average"} | Select -ExpandProperty Value

    $row."Avg Memory Usage (%)" = $memStat

    $diskStat = $_.Group | where {$_.MetricId -eq "disk.maxTotalLatency.latest"} | Select -ExpandProperty Value

    $row."Avg Disk Latency (ms)" = $diskStat

    $netStat = $_.Group | where {$_.MetricId -eq "net.usage.average"} | Select -ExpandProperty Value

    $row."Avg Net Usage (kb)" = $netStat

    $report += $row

}

}

$report | Export-Csv "C:\ReportData\VMPerfData.csv" -NoTypeInformation -UseCulture

[LucD]

I had similar experiences; when running Get-Stat in "bigger" environments it tends to consume a lot of memory and potentialy hang at some point.

That's most probably caused by the way the Get-Stat cmdlet is implemented, the vSphere PerformanceManager works  and/or the way PowerShell uses the stack and garbage cleaning.

There are a number of measures you can take:

• use my Get-Stat2 function, it consumes less memory. But in the end it will also come to a halt
• split the VMs in managable groups, instead of 1 Get-Stat call use several. The number to split at will depend on the environment and the memory size of the client where you run the script

The solution you picked takes of course the longest time to run, but probably uses the least amount of memory on the client.

[jlcjr2k8]

Hi Luc,

Is there a way to get this report in a graph like the pic above? I would like to present the report in graph format instead of numbers.

[LucD]

You can let Excel produce the graph, have a look at Export-Xlsx, the sequel, and ordered data.

And there are other ways, have a look at my Dutch VMUG: The Statistics Reporting Session

[jlcjr2k8]

Luc,

Both links are not working.

[LucD]

There was an error in the links. Still getting used to this new Jive version :smileyshocked:

Should be working now, can you try again ?

[jlcjr2k8]

Links working now.