DX Application Performance Management

Expand all | Collapse all

APM Setup is unstable

  • 1.  APM Setup is unstable

    Posted 03-09-2015 06:51 AM

    Hi All,

    We have a cluster APM setup with total of 20 Agents customized/configured in loadbalancing.xml to connect one collector. Hope worked fine for 10 days after we did the changes in loadbalancing.xml file.


    Issues found:

    1) Frequent disconnecting all agents.

    2) In Management Module, Metric Groupings, Dashboards and Alerts are not populating with data however when Agents are in connected status.


    Configuration changes done:

    Increased Java Heap Size in MOM, as well in two collectors.


    Observation from MOM Log:


    3/08/15 10:01:59.332 PM CAT [ERROR] [PO:WatchedAgentPO Mailman 2] [Manager.MessageService]
    Exception: java.lang.OutOfMemoryError: GC overhead limit exceeded calling method:
    {com.wily.introscope.spec.server.beans.agent.IAgentBridgeService.recordTimesliceBindingList, v1,


    I request anybody please suggest few valuable inputs.


    PFA for Log files from MOM Enterprise Manager and Collector.


    Swift response are highly appreciated.

    Many Thanks in Advance.

  • 2.  Re: APM Setup is unstable

    Broadcom Employee
    Posted 03-13-2015 10:01 AM

    Dear ShashiKrishna:


    There are two ways to look at these issues.


    Approach one: Fix instability. Then wait for next instability to occur and fix that one. Repeat as often as needed


    Approach two: Fix instability; Have quarterly healthchecks to optimize system so can handle present AND future needs. There are a variety of Tech Notes, KBs, and Tech Tips to help you do just that. It is important to have  a stable, scalable infrastructure


    Best of luck



    Hal German

  • 3.  Re: APM Setup is unstable

    Posted 03-13-2015 10:39 AM



    I strongly suggest you submit a CA support ticket; a quick look at your logs reveal a very unstable environment;

    - NTP sync problems

    - networks problems

    -very likely the collector smartstor is OOO.


    that's too much to offer a chance for not falling in OOM. or load balancer to work correctly.


    BR Marc

  • 4.  Re: APM Setup is unstable

    Broadcom Employee
    Posted 03-10-2015 04:21 AM
    Your EM ran out of memory.

    What is your current min and max heap settings?

    Are your running 32 or 64 bit OS?

  • 5.  Re: APM Setup is unstable

  • 6.  Re: APM Setup is unstable

    Posted 03-11-2015 01:25 AM

    Hi Davis,

    We have configured min and max heap size as 8GB i.e. 8192mb on MOM and 4GB on two collectors shows as below:



    lax.nl.java.option.additional=-Xms8192m -Xmx8192m -Djava.awt.headless=false -XX:MaxPermSize=256m -Dmail.mime.charset=UTF-8 -Dorg.owasp.esapi.resources=./config/esapi



    lax.nl.java.option.additional=-Xms4096m -Xmx4096m -Djava.awt.headless=false -XX:MaxPermSize=256m -Dmail.mime.charset=UTF-8 -Dorg.owasp.esapi.resources=./config/esapi


    My OS is 64 bit on MOM as well on two Collectors.




  • 7.  Re: APM Setup is unstable

    Broadcom Employee
    Posted 03-11-2015 01:57 AM

    What version of APM are you using?


    Add '-Xss512k' to each EM's JVM arguments.

  • 8.  Re: APM Setup is unstable

    Posted 03-13-2015 07:26 AM

    Hi Davis,

    Good day.

    We are waiting for your response since we are new to APM Implementation.


    Could you please direct us on your inputs to resolve APM setp stability problem.

  • 9.  Re: APM Setup is unstable

    Posted 03-11-2015 06:29 AM

    Hi Davis,

    We are using CA Introscope version.


    I request you to please detail the process to add the EM's JVM arguments.


    Thanks in Advance.




    +27 8140 83191

  • 10.  Re: APM Setup is unstable

    Broadcom Employee
    Posted 03-13-2015 09:17 AM
    Did adding the additional property help? This is a parameter that should have been added during the installation.