v9111 agent throwing Biz Definition ERRORs

[Legacy User]

On my testing v9111, on WAS70 with JDK 1.6 on AIX I get the following ERROR and WARN:

3/14/13 12:40:24 PM EDT [WARN] [IntroscopeAgent.Agent] Waited 15000 ms But did not receive the response for the message com.wily.isengard.messageprimitives.service.MessageServiceCallMessage: {com.wily.apm.spec.tess.beans.bizdefinition.IBizDefinitionLibrary.findComprehensiveTransactionComponents, v1, [[Lcom.wily.apm.spec.tess.beans.bizdefinition.SearchFilter;@230a230a]} from address Agent_189.client_main:269 to service address Server.main:342 from thread Agent Heartbeat -- We will not wait any longer
3/14/13 12:40:24 PM EDT [ERROR] [IntroscopeAgent.BizTrxDef] Error occurred and could not register Business Definitions
3/14/13 12:40:24 PM EDT [DEBUG] [IntroscopeAgent.BizTrxDef] Unable to register Business Definitions: Service call to host {Unknown} timed out after 15000 ms com.wily.isengard.messageprimitives.service.MessageServiceCallMessage: {com.wily.apm.spec.tess.beans.bizdefinition.IBizDefinitionLibrary.findComprehensiveTransactionComponents, v1, [[Lcom.wily.apm.spec.tess.beans.bizdefinition.SearchFilter;@230a230a]} threadname Agent Heartbeat
com.wily.isengard.messageprimitives.TimeoutConnectionException: Service call to host {Unknown} timed out after 15000 ms com.wily.isengard.messageprimitives.service.MessageServiceCallMessage: {com.wily.apm.spec.tess.beans.bizdefinition.IBizDefinitionLibrary.findComprehensiveTransactionComponents, v1, [[Lcom.wily.apm.spec.tess.beans.bizdefinition.SearchFilter;@230a230a]} threadname Agent Heartbeat

at com.wily.isengard.messageprimitives.service.MessageServiceClient.blockOnResponse(MessageServiceClient.java:282)

at com.wily.isengard.messageprimitives.service.MessageServiceClient.sendRequest(MessageServiceClient.java:163)

at com.wily.isengard.messageprimitives.service.MessageServiceClient.invoke(MessageServiceClient.java:356)

at $Proxy40.findComprehensiveTransactionComponents(Unknown Source)

at com.wily.introscope.agent.bizdef.BizDefAdministrator.registerAgentToEM(BizDefAdministrator.java:365)

at com.wily.introscope.agent.bizdef.BizDefAdministrator$1$1.connectionUp(BizDefAdministrator.java:299)

at com.wily.introscope.agent.connection.IsengardServerConnectionManager.addConnectionObserver(IsengardServerConnectionManager.java:427)

at com.wily.introscope.agent.bizdef.BizDefAdministrator$1.ITimestampedRunnable_execute(BizDefAdministrator.java:292)

at com.wily.util.heartbeat.IntervalHeartbeat$BehaviorNode.execute(IntervalHeartbeat.java:972)

at com.wily.util.heartbeat.IntervalHeartbeat.executeNextBehaviorAndCalculateSleepTime(IntervalHeartbeat.java:491)

at com.wily.util.heartbeat.IntervalHeartbeat.access$2(IntervalHeartbeat.java:445)

at com.wily.util.heartbeat.IntervalHeartbeat$HeartbeatRunnable.run(IntervalHeartbeat.java:668)

at java.lang.Thread.run(Thread.java:736)

Any idea how to resolve this?

Thanks in advance.

[Haruhiko Davis]

Hey Manish,
What is the full version of WAS?
What is the version of your JIT?

There's known performance issue with SR9. You should consider upgrading to at least SR10 or SR11. There were performance enhancements made to the Java compiler that did not make it into the IBM JRE until SR10.

[Legacy User]

Hiko,

WAS Version:

WVER0010I: Copyright (c) IBM Corporation 2002, 2005, 2008; All rights reserved.
WVER0012I: VersionInfo reporter version 1.15.5.1, dated 6/15/11

--------------------------------------------------------------------------------
IBM WebSphere Application Server Product Installation Status Report
--------------------------------------------------------------------------------

Report at date and time March 18, 2013 10:06:04 PM EDT

Installation
--------------------------------------------------------------------------------
Product Directory /usr/WAS70/AppServer
Version Directory /usr/WAS70/AppServer/properties/version
DTD Directory /usr/WAS70/AppServer/properties/version/dtd
Log Directory /usr/WAS70/AppServer/logs
Backup Directory /usr/WAS70/AppServer/properties/version/nif/backup
TMP Directory /tmp

Product List
--------------------------------------------------------------------------------
ND installed

Installed Product
--------------------------------------------------------------------------------
Name IBM WebSphere Application Server - ND
Version 7.0.0.23
ID ND
Build Level cf231218.02
Build Date 4/30/12
Architecture PPC64

--------------------------------------------------------------------------------
End Installation Status Report
--------------------------------------------------------------------------------

JIT Version:

java version "1.4.2"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2)
Classic VM (build 1.4.2, J2RE 1.4.2 IBM AIX build ca142-20080122 (SR10) (JIT enabled: jitc))

[Legacy User]

Hiko

Sorry, gave you the wrong JIT infor

#:/usr/WAS70/AppServer/java/bin$./java -version
java version "1.6.0"
Java(TM) SE Runtime Environment (build pap6460sr10fp1-20120321_01(SR10 FP1))
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc64-64 jvmap6460sr10fp1-20120202_101568 (JIT enabled, AOT enabled)
J9VM - 20120202_101568
JIT - r9_20111107_21307ifx1
GC - 20120202_AA)
JCL - 20120320_01

[Legacy User]

So, I have resolved half of the issue...............thanks partially to Hiko!

I commented out the 'biz-trx-http.pbd' in the 'websphere-typical.pbd' file. No one really uses this anymore and surprise why is it NOT commented out as a "out of the box" solution (like bizrecording.pbd).

Now, I can prob ignore the following WARN that I get in the logs.................

3/18/13 11:27:44 PM EDT [WARN] [IntroscopeAgent.Agent] Waited 15000 ms But did not receive the response for the message com.wily.isengard.messageprimitives.service.MessageServiceCallMessage: {com.wily.apm.spec.tess.beans.bizdefinition.IBizDefinitionLibrary.findComprehensiveTransactionComponents, v1, [[Lcom.wily.apm.spec.tess.beans.bizdefinition.SearchFilter;@55025502]} from address Agent_240.client_main:269 to service address Server.main:342 from thread Agent Heartbeat -- We will keep waiting and don't log further messages until we receive the reply or time out
3/18/13 11:27:44 PM EDT [WARN] [IntroscopeAgent.Agent] Got the response for the message com.wily.isengard.messageprimitives.service.MessageServiceCallMessage: {com.wily.apm.spec.tess.beans.bizdefinition.IBizDefinitionLibrary.findComprehensiveTransactionComponents, v1, [[Lcom.wily.apm.spec.tess.beans.bizdefinition.SearchFilter;@55025502]} to address Agent_240.client_main:269 service address Server.main:342 from thread Agent Heartbeat

If that is true then I am ready to certify my testing and move on with deployment plan, cool)

Thanks
Manish

[Haruhiko Davis]

Are there any other events that occur around the same time?