Cannot get custom.pbd file recognized

[Anon Anon]

I was told by ca support to post this here since they do not support custom.pbds. 

 

Situation:

 

I have an "APP" folder showing on one agent and now the client wants this to appear on all agents for continuity.

 

Steps taken:

I have added the path to the InstrscopeAgentprofile so it is identical to the other agents profile,

I have added the custom.pbd file for this made sure the spelling was correct

It still is not getting picked up when server is recycled.

 

Suggestions?

[reian06]

Please check the AutoProbe.log for the VM where the metrics are not showing:

 

a) Check the pbd file is listed in the AutoProbe.log

b) Check the classes instrumented in custom.pbd are listed in the AutoProbe.log (Second part of the AutoProbe.log)

c) Check the classes / methods from custom.pbd if the methods are listed as instrumented with the tracer

d) Metrics are still not showing

 

Error analysis:

a) fails => The pbd file was not accessible, not there or the restart didn't happen. Maybe wrong IntroscopeAgent.profile ?

b) fails => The code which is targeted on "custom.pbd" was not (yet) executed / not loaded

c) fails => either the class doesn't declare the method or it's declared in a super class and thus the byte code injection failed.

d) fails => Check whether you run on the Agent into a metric clamp situration

 

Small side node. In case it's on WebSphere's with Class Caching enabled, strange things can happen with Byte Code Instrumentation. Mostly once a change is done on injection, it is recommend to clear the class cache and restart all nodes.

[Anon Anon]

Servers have been recycled and there is no Autoprobe.log file appearing for this server.   The way it is setup each server has its own Autoprobe.log file stored on a file share in another location ie: \\ho\***\wily\introscopeHPUXefile_psps_AutoProbe

The only files shown in the logs file location being updated are:

efile_psps_AutoProbe

efile_psps_IntroscopeAgent

 

Which is the server and agent we are trying to replicate

[Anon Anon]

Below is the custom pbd that is being called in the profile

 

# DI Case Contact Clases.pbd written by YATT on Thu Oct 12 16:03:47 CDT 2006

######################
# default
# ====================
# section order no: 0
# Default section created by YATT.
TraceAllMethodsOfClass: com.nm.ulife.printingservice.barcode.BarCodeService  BlamedMethodTimer "APP|{classname}|{method}:Response time (ms)"
TraceAllMethodsOfClass: com.nm.ulife.printingservice.barcode.BarCodeService  BlamedPerIntervalCounter "APP|{classname}|{method}:per Interval Counter"

TraceAllMethodsOfClass: com.nm.ulife.printingservice.process.IPrintingProcessAccess  BlamedMethodTimer "APP|{classname}|{method}:Response time (ms)"
TraceAllMethodsOfClass: com.nm.ulife.printingservice.process.IPrintingProcessAccess  BlamedPerIntervalCounter "APP|{classname}|{method}:per Interval Counter"
TraceAllMethodsOfClass: com.nm.ulife.printingservice.process.IdentifierSheetCreator  BlamedMethodTimer "APP|{classname}|{method}:Response time (ms)"
TraceAllMethodsOfClass: com.nm.ulife.printingservice.process.IdentifierSheetCreator  BlamedPerIntervalCounter "APP|{classname}|{method}:per Interval Counter"
TraceAllMethodsOfClass: com.nm.ulife.printingservice.process.PrintingProcessAccessBean  BlamedMethodTimer "APP|{classname}|{method}:Response time (ms)"
TraceAllMethodsOfClass: com.nm.ulife.printingservice.process.PrintingProcessAccessBean  BlamedPerIntervalCounter "APP|{classname}|{method}:per Interval Counter"
TraceAllMethodsOfClass: com.nm.ulife.printingservice.process.PrintingProcessAccessHome  BlamedMethodTimer "APP|{classname}|{method}:Response time (ms)"
TraceAllMethodsOfClass: com.nm.ulife.printingservice.process.PrintingProcessAccessHome  BlamedPerIntervalCounter "APP|{classname}|{method}:per Interval Counter"

[reian06]

Yes, please go through the steps of my previous post...

[Viruvaibhav_apm]

Hi,

 

Could you please share\attach the AutoProbe.log for the instance\Server where "APP" folder is not turning up ?

 

Secondly, are you sure that profile file which you are editing is been mentioned in the JVM arguments to that server ?

 

regards,

Vaibhav

[Anon Anon]

AutoProbe.log not being updated at all. 

[reian06]

Perfect.. so you are at "a) fails" or WebSphere class caching. I don't know your environment ..

[Anon Anon]

Yes Websphere

[Viruvaibhav_apm]

Indicating no instrumentation happening out of that custom pbd.

 

Now could you please verify that profile file you are updating is the one you have mentioned in the JVM arguments ?

 

Regards,

Vaibhav

[Anon Anon]

Yes it is

[Anon Anon]

I just put "custom" to mask what the call was

[Viruvaibhav_apm]

It seems to me that your updated profile file is not being picked up when u r recycling the server otherwise atleast 'default-full.pbl' would have been picked up and showed up as in the AutoProbe.log

If possible, could you please check in the Introscope.log and see after resart anything unusual you are observing like errors\exception.

[Anon Anon]

Yes but that won’t be until Monday am since they recycle at 03:00.

 

Thanks for your help – it is much appreciated

[Anon Anon]

[Hallett German]

Hi Patrick:

At this point, my I suggest opening a case to get additional assistance.

 

Thanks

Hal German

APM SUpport

[Anon Anon]

“Ca does not support custom pbds” go to the community as per support

[Michael Cook]

Hi,

 

Whilst that is correct we do not support customization – the fact that you are not getting any logs produced is something we should be able to help with and that in turn may help point you in the right direction with regard to why your custom pbd is not working as expected.

 

Thanks

Mike

[Viruvaibhav_apm]

As suggested by Mike, you can open ticket that your AutoProbe and agent logs are not getting created.

 

Meanwhile, I see some AutoProbe file is been created on 22 July. Is that related to same application ? If yes, then that is the day when you had made changes to include custom pbd ?

 

Also I see a "IntroscopeAgent" file in the directory. Is it the same you have edited to bring custom pbd for your applcation ? [Pre-assuming as seeing the date as 22 july]
If yes, then I see the extension of this file as some application specific "uulife file" and not a Profile file. Can we just cross check the extension as well as the profile file location ?

 

Regards,

Vaibhav

[Anon Anon]

We think we have found the culprit - permission issues

7/27/15 03:20:23 AM CDT [WARN] [IntroscopeAgent.Agent] The Introscope AutoProbe log file /nmlprod/wily/introscopeHPUX/logs/uulife_AutoProbe.log could not be created: /nmlprod/wily/introscopeHPUX/logs/uulife_AutoProbe.log (Permission denied (errno:13))  No logging will occur.

 

We will know more tomorrow am after reboot.

 

I'd like to thank everyone for their input on this.

[Viruvaibhav_apm]

Great....
Actually I was about to ask you this only when I wrote - "..as well as the profile file location ?"... But I missed that

 

But good that something you got through. Let us know more details once you get it done.

 

Regards,

Vaibhav

[Hallett German]

Dear Patrick:

     That great to hear. Usually when logs are not being created and the account is correct, I point them to a new directory with no permissions or temp just to see if it is a permissions issue. If that does not resolve it, please feel free to open a Support case just on the "no logs" aspect.

 

Thanks

Hallett German

APM Support

[Anon Anon]

Introscopeagent.uulife is listed as an agent profile specifically for the uulife app

[reian06]

Patrick, just to be sure. We are still at the beginning? The pbd file is not listed in the AutoProbe.log?

[Anon Anon]

It is listed in the AutoProbe.log

[reian06]

Perfect, here again our check list:

 

a) Check the pbd file is listed in the AutoProbe.log

b) Check the classes instrumented in custom.pbd are listed in the AutoProbe.log (Second part of the AutoProbe.log)

c) Check the classes / methods from custom.pbd if the methods are listed as instrumented with the tracer

d) Metrics are still not showing

 

Error analysis:

a) fails => The pbd file was not accessible, not there or the restart didn't happen. Maybe wrong IntroscopeAgent.profile ?

b) fails => The code which is targeted on "custom.pbd" was not (yet) executed / not loaded

c) fails => either the class doesn't declare the method or it's declared in a super class and thus the byte code injection failed.

d) fails => Check whether you run on the Agent into a metric clamp situration

 

Looks like we are past a).

[Anon Anon]

Log files are now being written.  It was a permissions issue.   Unfortunately the pbd that we want to see is still not getting picked up. ulifeservice.pbd  .   Just in case there is an issue with capitalization we have made everything lower case since this is running on HPUNIX.

[Anon Anon]

7/29/15 10:20:02 AM CDT [ERROR] [IntroscopeAgent.Agent] Introscope AutoProbe will not run because the provided directives files are not valid: Invalid directives entry UlifeService.pbd found in directives list file /nmlprod/wily/introscopeHPUX/default-full.pbl Please restart with valid directives

[reian06]

OK, that looks better. Looks like you have the UlifeServe.pbd in the directives and inside the pbl file. you can remove it at one place and you have to take care about the lower/upper case.

[Anon Anon]

UPDATE:  working with CA support.  uulife.pbd was being called from 2 locations.   Commented out the file in the pbl directory - now have to wait until recycle to see if this works.

[Anon Anon]

Errors are now gone but we still do not see the APP

[reian06]

Patrick,

 

coming back to our initial check list. At which item are we now?

 

Thanks

Andreas

[Anon Anon]

Having app team check and verify step b  making sure that the com.nm.ulife.printingservice.process.PrintingProcessAccessHome  exists.

[Anon Anon]

This was the issue - this was not the correct path and app person was on vacation.  Thanks to all for your input