Gone are the days of milkmen when you left numerous bottles outside your door. Almost magically in the early hours of the dawn back then, the old milk bottles disappeared only to be replaced with new ones.
The same sort of magic also happens with APM CE (CEM) Services. Each morning when arriving at work, there are new incidents and reports, updated statistical tables, and a cleaner database. (Note: If monitoring too many transactions, creating too many defects, or not following best practices on server requirements/architecture, then the above results may or may not happen.)
So let's talk about what is going on behind the scenes to get the daily report:
The TIM or MTP/TIM generates the needed ingredients of the recipe to make reports. This includes defects, statistics, and session/login mapping information (in cache). Looking at the timfiles log shows which files are or are not being processed. Most of the interim files are stored under /etc/wily/cem/tim/data/out. The stats files are kept for the previous hour while the other files are collected every five seconds.
The TIM Collector pulls the files off of the TIM. It also does the hourly or interval aggregation of statistical information. (This collector additionally does RTTM and Transaction Discovery processing. Because it is busy around the clock, a dedicated TIM Collector server is highly recommended.) If for some reason the TIM Collector gets bogged down in processing, then data may back up (But not lost.) on the TIM. A restart of the TIM Collector typically resolves this issue.
The Stats Aggregation Collector Service runs by default shortly after midnight. It aggregates the hourly into weekly, weekly into monthly, and monthly into yearly data. These processes run independently of each other. Up to two years of data can be processed. Depending how many defects that you have, your configuration settings, and whatever else that server is doing, this may take minutes or hours. Viewing the log in INFO not DEBUG mode allows viewing the processing over several days. Many of the settings for this are configurable in EM-HOME\config\tess-default.properties\tess-customer.properties. (Or equivalent location. There are other files in that directory that can also be configured to impact performance.)
The APM Database Cleanup Collector Service is the least intensive service and typically runs on the Stats Aggregation Collector. At three A.M. it kicks off vacuuming and soft deleting old records and tables.
The APM Database stores the output from the TIM in Postgres and Oracle database tables. If available, it also stores Appmap data. Without proper maintenance, it can grow quite large and can impact performance in displaying reports. Typically large tables are users, user groups, session maps, app maps, and defect metadata.
The MOM GUI is used to display and to schedule the creation of reports. But it is dependent on all the above severs to do their jobs before a result is shown. So keep in mind all of the magic involved when looking at your next report.
Thanks to Safekidda and Wikimedia.org for use of the image.