DX Application Performance Management

  • 1.  introscope agent on Weblogic 10 w/t 1.5.0_18 JVM

    Posted Feb 04, 2011 05:30 PM
    hello list,
    seems like i have hit a road block with this config...
    i am trying to install APM90 introscope agent on Weblogic V 10 with 1.5.0_18 (32 bit) JVM; server startsup fine but then 3-4 mnts later it crashes. i have a similar setup in LAB with 3 instances on the same server with same weblogic and JVM versions... and these LAB servers are running JUST fine.

    CA Support says that cause is JVM 1.5.0_18 and they recommend upgrading it... that's a little difficult as that in itself will be a 9 -12 mnth project :grin:

    any Wily user out there who has this config and have APM 90 agent working on them?

    thanks
    ashish


  • 2.  RE: introscope agent on Weblogic 10 w/t 1.5.0_18 JVM

    Broadcom Employee
    Posted Feb 07, 2011 04:56 PM
    Hello Ashish,

    Jvm crashes has to do with a defect/bug in the jvm as it is not suppose to crash under any circumstance, therefore you should open a suppor ticket with the jvm provider.

    In the meantime, you could try:

    1. Use an alternate JVM (Sun Java 6 beta, IBM jdk, etc).

    2. Configure the Appserver with -Xbootclasspath instead of -javaagent switch.

    3. Disable SQLAgent by removing the AGENT_HOME/ext/SQLAgent.jar out of the AGENT directory - the problem could be related to a metric explosion.

    4.Disable Platform monitor: move the following files from the /wily/ext directory to another directory.
    As far as I know, PM component it is the only one that could make native calls and could possible cause this issue, but I doubt.

    5. For testint purpose, disable JMX collection. Set introscope.agent.jmx.enable=false
    Polling lots of JMX metrics is CPU intensive. If possible, set for now introscope.agent.jmx.enable=false

    6. Find out if the problematic server is at the same OS level patch as the other similar setups (LABS).

    Kindly understand that this is an underlying JVM problem and there is not much we can do from our end to fix this, other than suggesting the above workarounds.

    Regards,

    Sergio


  • 3.  RE: introscope agent on Weblogic 10 w/t 1.5.0_18 JVM

    Broadcom Employee
    Posted Feb 08, 2011 02:21 PM
    It is recommended by Oracle to use Sun's JRE instead of JRockit for large-scale deployments.
    With that in mind, I'd follow Sergio's suggestion and replace your JRE first before getting too deep and see if this helps to stabilize the application.

    If you are experiencing a SQL metric explosion, then add the following to your agent profile to limit the query length captured:

    introscope.agent.sqlagent.sql.maxlength=*** where *** equals a number between 100 - 999. By default, the agent captures 999 characters, so start at five or six hundred.

    If you're using Oracle for your database, make sure to remove the Oracle PP from your Java agents. There's a known issue with the PP and monitoring Oracle databases with OEM. I'm not sure if the problem exists with APM, but it does with Introscope 8.

    -Hiko


  • 4.  RE: introscope agent on Weblogic 10 w/t 1.5.0_18 JVM

    Posted Feb 09, 2011 08:33 AM
    the app that is deployed on these webserver is a third party and apps developers are reluctant to change JVM version.. well let's just it will tkae 4-6 months to re-qualify this app with new JVM :(

    i have tried -Xbootclasspath already, still crashes. havent tried removing SQL and Platform stuff. will try that and update this post..


    Thanks
    Ashish


  • 5.  RE: introscope agent on Weblogic 10 w/t 1.5.0_18 JVM

    Broadcom Employee
    Posted Feb 17, 2011 08:06 PM
    Another suggestion for your troubleshooting is to remove all PowerPacks you may have installed on the agent.
    If the JVM can run without the PowerPacks, then you know it's one of those. If not, then's it's a JRE issue. Will you still run into a QA issue if you move to a more updated version of JRE 1.5?
    If you're running any version of Introscope 7-8, make sure you don't have the Leak Hunter jar file in your agent. Even though you may have set the enabled value to false your profile, the agent will load the jar file into memory. Either remove the jar from AGENT_HOME/ext or rename it.

    -Hiko


  • 6.  RE: introscope agent on Weblogic 10 w/t 1.5.0_18 JVM

    Posted Apr 15, 2011 04:23 PM
    Hi Sergio,

    JDK is crashing intermittently in weblogic 9.2MP3 with sun JDK 1.5.0_17 after introscope(9.0.6) installation,crash file shows its failing at following task.Please help to find what would be the reason

    Current CompileTask:
    opto: 55% com.wily.introscope.api.instrument.AClassRedefinitionDelegate.getInstrumentableClass(Ljava/lang/String;Ljava/la
    ng/ClassLoader;)Ljava/lang/Class; @ 65 (160 bytes)


  • 7.  RE: introscope agent on Weblogic 10 w/t 1.5.0_18 JVM

    Posted Apr 19, 2011 03:51 PM
    for what it's worth, app team had agreed for JVM upgrade to 1.5.0_22; we are using Introscope 9.0.0 agents; we will upgrade to 9.0.6 soon. will report if the newest agent still works with no issues,

    just for the records we were trying to instrument the old JVM (1.5.0.18) with WL 10.0 using Introscope 9.0.0 agents.

    thanks
    ashish


  • 8.  RE: introscope agent on Weblogic 10 w/t 1.5.0_18 JVM

    Broadcom Employee
    Posted Apr 20, 2011 05:09 PM
    Just remember to try to use "-javaagent:" first to instrument the JVM; then "-Xbootclasspath" like Sergio suggested.


  • 9.  RE: introscope agent on Weblogic 10 w/t 1.5.0_18 JVM

    Posted Apr 22, 2011 11:35 AM
    Hi,

    Right now iam using javaagent and iam seeing above crash.
    1)Should i use both javaagent and xbootclasspath?
    2)Is it recommended to use xbootclasspath for jdk 1.5 or higher.?
    3)Do usage of -xbootclasspath cause any reduced metrics collection or malfunctionality?

    Thanks in advance