How to keep detailed metric data for more than 7 days?

[boris_hyperic]

Hi,

I need to see trends of CPU and memory usage over a long period of time. Data Manager Configuration UI allows to keep metric data for only up to 7 days (administration -> HQ Server Settings -> Delete Detailed Metric Data Older Than).

Is there a way to preserve data for several weeks or months?

Thank you

[edan_hyperic]

Have you just started using HQ? It keeps hourly averages (including highs) for a very long time. The 7 day limit is only how long it keeps the raw data, such as 5 minute samples of CPU.

I find the averaged data perfectly acceptable for viewing trends over a long period of time--you can't see 12 samples per hour over 3 months anyway :)

Now, if you REALLY need to keep the raw data, you can look at http://support.hyperic.com/display/DOC/Building+a+Metric+Data+Warehouse. However, then you'll have to write your own queries against the database, build your own reports, etc. Maybe Hyperic's "Operations IQ" or whatever its called would make that easier.

But, I would suspect you would be fine with the summary data that HQ already keeps for months and months :)

[sylvestersteele]

cite="mid:[e-mail]"
type="cite">

But, I would suspect you would be fine with the summary data that HQ already keeps for months and months :)

I am having problems with something similar. We have 2 HQ servers
running, and they retain data for different periods of time. We haven't
yet found a way to fix this. My post is at class="Apple-style-span"
style="border-collapse: separate; color: rgb(0, 0, 0); font-family: 'Times New Roman'; font-size: 16px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;"> class="Apple-style-span"
style="font-family: Tahoma; font-size: 12px; white-space: pre;">http://communities.vmware.com/message/1934000#1934000&#24000.



Sylvester

[Dans_hyperic]

HQ don't have this feature (i don't found it). But you could add it with a batch SQL that "copy" the data from last day to "another" table.. didn't?

[boris_hyperic]

Thanks for helpful replies. I now see tables with aggregated data and I agree it is not difficult to copy detailed measurements to a different table.

However already implemented indicator graphs are very nice and I was hoping there is an easy way to see 2-4 weeks worth of data on the same graph. I don't have too many measurements and enough space in the DB. Well, maybe a longer range will be allowed for future release...

Regards

[edan_hyperic]

> However already implemented indicator graphs are very
> nice and I was hoping there is an easy way to see 2-4
> weeks worth of data on the same graph. I don't have
> too many measurements and enough space in the DB.
> Well, maybe a longer range will be allowed for future
> release...

I don't understand... I can pull back, say, Nov 1st 2008 to Dec 31st 2008 in the default Indicators views very easily by using "Edit Range." It looks like there are some threads where people have problems, but I'm not aware of HQ *ever* deleting old average data. I have data from some agents from last May.

Maybe you just missed the "Edit Range" option? Unless I'm confused, I think it already does everything you are asking for :)

[boris_hyperic]

I know 'Edit Range' but older data is very coarse and I wanted to see how boxes handle peak 1-2 hours. Problems usually occur during peak hours and relatively little of interest happens over the rest of the day. So daily averages are good to have but are of limited value.

Thank you

[edan_hyperic]

Oh, ok, I think I got it. If you go back two weeks, you can't see hourly data anymore for one day, two weeks ago. I have thought I wanted that before, too. The notion that I might want to see highly granular data for a historical event is why I have set my HQ to store 7 days (max) of detailed data.

However, it does still store a couple peak/avg/low for each day two weeks and further back. To your concern about performance during peak hours, you would have data points showing maximum cpu/mem/etc. Not perfect, but ...

... Personally, I find that if I'm interested in performance at almost at any given time, I'm interested in how the performance compares *in context.* I want to see how last week performed in reference to the week before, and so far today. So in practice, I have actually never desired that kind of granularity, like hourly data from 8AM-8PM this day last month. I never look at it that way, even though I have sometimes thought I would.

In fact, in my previous life, as a dedicated "performance engineer," I developed a strategy for storing detailed data from several proprietary systems in a single MySQL table. It istill used today, and it has perhaps 100 million rows and the index is as big as a data :) Point being, I built that more than 1 or 2 years ago, and I have never written a query to show me 5 minute data points for "2007-06-05 13:05" to "2007-06-05 15:15" :)

Now, that's not to say that it wouldn't be cool if HQ had a more easily configured data archive feature, maybe with an option to use Infobright Community Edition (http://www.infobright.org/) for a neat compressed warehouse of the data :) But until I have time to fool with that and an archive of HQ's data... I'll probably set my detailed data range down to 3 or 4 days, just so I have detailed data on Monday for something weird on Friday :)

Cheers.

[dr7d7]

Hi,
I have what may be a related problem. I have my settings set to store 7 days of detailed data as well. When I set the chart range to "last 7 days" the chart works, however, when I set the chart to "last 8 days" I get an "There are no indicator metrics data to display for time period between.." error. After reading this thread, I thought that it might be that the compressed data points are only actually occurring once a week. So I tried "last 15 days" and "last 30 days". Still no data. I'm beginning to think the data is being deleted (and not compressed).

Please see my other post:
http://communities.vmware.com/thread/353782?tstart=0

Thanks in advance