CA Wily Technical Advisory: Agent to EM communication state breakdown may cause memory leak
CA Wily has uncovered a potential issue where the Introscope Agent may fall into a bad communication state and slowly leak memory. This technical advisory describes the affected versions and platforms in more detail and explains what remediation actions are recommended.
Problem Description
CA Wily has identified a scenario where the Agent enters into a state where it believes the Enterprise Manager is up and communicating but it is not. In this state there is at least one Vector that slowly builds up data. This may eventually cause the application under management to run out of memory.
Please note that this problem is exacerbated for environments that run the 8.1 MQ PP as there is an additional ArrayList collection which may add data that it may be unable to release.
Problem Symptoms
Agent Side Memory Levels
If there is a communication breakdown and the Agent ends up experiencing this issue, there will be an observable increase in memory levels and GCs for monitored applications in the Workstation.
Introscope Agent Log File:
The Agent log should have evidence that the Agent is stuck in the problematic communication state. If the 1st message below appears without either of the next two for more than a few minutes, then the Agent has likely entered an indefinite blocking state.
•
Waited 120000 ms But did not receive the response for the message com.wily.isengard.messageprimitives.service.MessageServiceCallMessage:.....
-
Got the response for the message com.wily.isengard.messageprimitives.service.MessageServiceCallMessag
-
Enterprise Manager responding too slowly, disconnecting
Who is Impacted
• These are the affected Introscope Agent versions:
o 8.0.0 - 8.0.3, 8.1.0 – 8.1.1, and 8.2.0 – 8.2.1
*Note: Some sub-patches of the versions listed above have intermediary fixes, but our final fix is planned for the delivery vehicles listed below
• The applications under management should not be subject to this issue if:
o You are using the Introscope 8.0.4+ agent
o You are using the Introscope 8.1.2+ agent
o You are using the Introscope 8.2.2+ agent
o You are using the Introscope 9.0+ agent
o You are a CEM only customer
**Note: The timing issues involved here are extraordinarily difficult to encounter and reproduce. We believe 7.x Agents will not encounter this situation due to simpler thread architecture.
Solution Timing and Workaround
• This issue will be addressed in the following Introscope agent versions:
o Introscope 9.0: this issue does not occur with 9.x agents
o Introscope 8.2: there was a change in our 8.2.2.0 release such that this issue no longer occurs
o Introscope 8.1: there was a change in our 8.1.2.0 release such that this issue no longer occurs
o Introscope 8.0: there was a change in our 8.0.4.0 release such that this issue no longer occurs
• In the meantime, if you suspect your Agent is in this bad communications state you may be able to free it by explicitly terminating your Enterprise Manager.
For More Information
If you have any questions or concerns, please contact CA | Wily Technology Technical Support at
support@wilytech.com, or at any of the following numbers:
•
Toll-Free U.S.: 1 888 GET WILY ext. 1
•
U.S.: +1 630 505 6966
•
Latin America +55 11 5503 6167
•
Europe: +44 (0)1753 242763
•
Asia-Pacific: +81 3 6868 2300
•
Toll-Free Japan: 0120 974 580
•
Singapore: +65 6432 8641
•
Australia: +61 2 8898 2963
•
Call any Ca Technologies office and ask for xtn 46966
To receive automatic notification of any patches or fixes, please subscribe to Knowledge Base article 498, available at
http://support.wilytech.com/cgi-bin/wilytech.cfg/php/enduser/std_adp.php?p_faqid=498.