Getting Peak Utilization with get-stat

[TheVMinator]

In vCOps you have the concept of the 95th percentile, which you can use to get "average peaks".  The idea is that you are taking a set of data points and getting the 95th percentile of them for something like CPU demand in MHz for a set of VMs.

Transfer to PowerCLI - in powerCLI you are relying on the vCenter Database data points which have been averaged through rollups.

The problem is this:

If I am using only data points from the last hour of "real-time"statistics, my 95th percentile is done with data points representing the 20 second polling intervals.

When I move to "Past Day" data points, then I am using averages of those 20 second polling points to get just data points representing averages of the 5 minutes intervals.  In other words, I've already lost some of my high and low data points in the rollup and the per-5 minute data points are now smoothed out to be averages of the 20 second intervals I had.

Then a week passes and I have rollups of my "Past Day" data points.  Now my data points are smoothed out even further.  Among the set of data points left now, I've lost some more of the highs and lows in my 5 minute intervals, and I only have the averages of those 5 minute intervals in my 30 minute interval data points.

The point is that the farther I go back, the more peaks and valleys I lose as the rollups tone them off.  So If I do a 95th percentile calculation, the older the time period I'm reporting on, the less of a true representation of 95th percentile I really have.  Because I am working off data where the peaks and lows have been rounded more and more to get fewer data points.

Thanks to LucD, who proposed a way to get 95th percentile:

https://communities.vmware.com/thread/460129

I can get a rough estimate of the 95th percentile.  However it seems (correct me if wrong) that as I go back a month, or two or three months or a year that my 95th percentile is going to be less and less of an acutal 95th percentile and more and more a 95th percentile of averages instead of a 95th percentile of actual performance data. and probably further from what vCOps would report.

Any proposed solutions to this issue?  The goal here is to be able to use PowerCLI to truly model what my "peak demand" periods look like for a VMs CPU demand when for example at the end of the month everyone does their processing.

[LucD]

Yes, that is the disadvantage of the aggregation that takes place on the counters.

Afaik, there are 2 solutions to this problem (but neither will bring the "lost" granularity back I'm afraid).

1) Run your report daily (on the 20 second intervals) and store the reports for future reference.

2) The vCenter aggregation is done with a purpose. Keeping the 20 second interval data for a year would represent a huge amount of data.

An alternative is to extract a limited set of metrics (the once you know you will need later on) on a daily basis and store them in another database.

In my upcoming book, I'll discuss both methods in more detail.

[TheVMinator]

vCOps has an adapter to pull information out of it.   The vCOps info is stored longer and with more detail than in vCenter. Is it possible to begin to use that as a means to be able to script against data that isn't in vCenter anymore?  Eventually it seems like we need to be able to hit vCOps database with powerCLI not just vCenter....perhaps this is a preliminary step?

[LucD]

Although there is no official/supported method to pull data from the vCOps DB, it is possible.

Clinton did a post on that, see Unofficial VMware vCenter Operations Powershell Module

And needless to say, there will be a vCOps chapter in my upcoming book I mentioned earlier :smileywink:

[TheVMinator]

OK great.  I see the release date is end of May.  Hopefully vCOps will be powerCLI extensible before then and we can read all about it, or maybe in version 2 of your book.  It does seem like vCOps accessibility via PowerCLI is going to change the reporting game drastically when it comes.