Script Plugin Help

[tegulizard13]

I have created a perl script to give me process information via top (just as a practice plugin, not really important) And I can't seem to get it working.
It returns information in appropriate key=value pairs, and I have created an xml descriptor which I suspect I have done wrong.

The key value pairs display as such:
Availability=0
number=107
sleeping=106
running=0
oncpu=1

And my xml file looks like this:
<plugin>
<server name="procmon">
<filter name="script" value="../scripts/proc.pl"/>
<config>
</config>
<plugin type="measurement" class="org.hyperic.hq.product.MeasurementPlugin"/>
<service name="Processes">
<config>
</config>
<filter name="template"
value="exec:file=%script%"/>
<metric name="Availability"
templare="${template}:Availability"
indicator="true"/>
<metric name="Number of Processes"
category="UTILIZATION"
indicator="true"
template="${template}:number"/>
<metric name="Number of Sleeping Processes"
category="UTILIZATION"
indicator="true"
template="${template}:sleeping"/>
<metric name="Number of Running Processes"
category="UTILIZATION"
indicator="true"
template="${template}:running"/>
<metric name="Number of Processes on the CPU"
category="UTILIZATION"
indicator="true"
template="${template}:oncpu"/>
</service>
</server>
</plugin>

Again, I can't seem to get this working, where have I gone wrong?

Thanks

[excowboy]

Hi,

what is your error message or your agent.log output ? Your plugin seems to be syntactically correct.
I had some problems to get a script plugin running and adding this property within the server section solved the problem:

<property name="BACKGROUND_COMMAND" value="false">

Cheers,
Mirko

[tegulizard13]

My apologies for responding so late, I haven't had a chance to check the log file or try your suggestion, but I will post the results here within the next couple of days.

Thanks

[tegulizard13]

Here is the relevant part of my agent.log:

2009-06-10 13:46:42,214 INFO [Thread-0] [ProductPluginManager] Loading plugin: proc-plugin.xml
2009-06-10 13:46:42,264 INFO [Thread-0] [ProductPluginManager] Loading plugin: proc-plugin.xml
2009-06-10 13:46:42,300 ERROR [Thread-0] [AgentDaemon] Plugin initialize > 1 time
org.hyperic.hq.product.PluginExistsException: Plugin name=proc already exists
at org.hyperic.hq.product.PluginManager.registerPlugin(PluginManager.java:354)
at org.hyperic.hq.product.ProductPluginManager.registerPluginJar(ProductPluginManager.java:1011)
at org.hyperic.hq.product.ProductPluginManager.registerPluginJar(ProductPluginManager.java:743)
at org.hyperic.hq.product.ProductPluginManager.registerPlugins(ProductPluginManager.java:831)
at org.hyperic.hq.product.ProductPluginManager.registerCustomPlugins(ProductPluginManager.java:774)
at org.hyperic.hq.agent.server.AgentDaemon.startPluginManagers(AgentDaemon.java:684)
at org.hyperic.hq.agent.server.AgentDaemon.start(AgentDaemon.java:827)
at org.hyperic.hq.agent.server.AgentDaemon$RunnableAgent.run(AgentDaemon.java:925)
at java.lang.Thread.run(Unknown Source)
2009-06-10 13:46:42,304 ERROR [Thread-0] [AgentDaemon] Unable to initialize plugin manager: Plugin name=proc already exists
org.hyperic.hq.agent.server.AgentStartException: Unable to initialize plugin manager: Plugin name=proc already exists
at org.hyperic.hq.agent.server.AgentDaemon.startPluginManagers(AgentDaemon.java:689)
at org.hyperic.hq.agent.server.AgentDaemon.start(AgentDaemon.java:827)
at org.hyperic.hq.agent.server.AgentDaemon$RunnableAgent.run(AgentDaemon.java:925)
at java.lang.Thread.run(Unknown Source)
2009-06-10 13:46:42,339 INFO [Thread-0] [AgentDaemon] Agent shut down
2009-06-10 13:46:42,339 ERROR [Thread-0] [AgentDaemon] Agent startup error:
org.hyperic.hq.agent.server.AgentStartException: Unable to initialize plugin manager: Plugin name=proc already exists
at org.hyperic.hq.agent.server.AgentDaemon.startPluginManagers(AgentDaemon.java:689)
at org.hyperic.hq.agent.server.AgentDaemon.start(AgentDaemon.java:827)
at org.hyperic.hq.agent.server.AgentDaemon$RunnableAgent.run(AgentDaemon.java:925)
at java.lang.Thread.run(Unknown Source)
2009-06-10 13:46:42,340 INFO [Thread-0] [AgentDaemon] Attempting to rollback agent bundle
2009-06-10 13:46:42,364 INFO [Thread-0] [AgentDaemon] Rollback of agent bundle was successful
2009-06-10 13:46:42,364 INFO [Thread-0] [AgentDaemon] Restarting JVM...
2009-06-10 13:46:43,382 INFO [Wrapper-Restarter] [AgentConnection] 127.0.0.1:2144 -> agent:die
2009-06-10 13:49:13,679 INFO [main] [AgentConnection] 127.0.0.1:2144 -> agent:ping
2009-06-10 13:49:13,924 INFO [WrapperStartStopAppMain] [AgentConnection] 127.0.0.1:2144 -> agent:ping
2009-06-10 13:49:16,970 INFO [main] [AgentConnection] 127.0.0.1:2144 -> agent:ping
2009-06-10 13:49:17,980 INFO [main] [AgentConnection] 127.0.0.1:2144 -> agent:ping

I can add the server and a service for that server, but no metrics show up as options for any of it.

[jvalkeal_hyperi]

Hmm, its loading your proc-plugin.xml twice. Any change that you have a duplicate somewhere?

[tegulizard13]

I have one in hyperic/hq-plugins, and one in hyperic/hyperic-hq-agent-4.0.3/bundles/agent-4.0.3-953/pdk/plugins/

[jvalkeal_hyperi]

Yes, that extra message is coming because of that. You should only store it to one location. You can choose which one to use. I prefer hyperic/hq-plugins for custom stuff.

btw, is that snipped the one you're using. Just noticed that there is a typo:

<metric name="Availability"
templare="${template}:Availability"

templare should be template

[tegulizard13]

I actually noticed that typo soon after initially posting and fixed it, thank you though. I will remove the plugin from one of the locations and find out if it's still broken.

[jvalkeal_hyperi]

Then do you know if your perl script is actually executed since you're using relative path. I would make this sure by using absolute path.

Try to test your plugin using dry-run:
http://support.hyperic.com/display/DOC/Invoking+Plugins+Standalone

It's much easier way to test your plugins during development.

[tegulizard13]

I have tried both the relative and normal paths to the script since the original post, and I can't seem to generate metrics. What commands should I be issuing?
I tried:
java -jar hq-product.jar -p proc -a metric
java -jar hq-product.jar -p proc -m metric
and some others

[jvalkeal_hyperi]

DEBUG [CollectorThread] [Collector] name=exec, thread=CollectorThread, result=Fri Jun 12 09:40:24 EEST 2009 [Info] (/tmp/proc.sh) Availability=0
number=107
sleeping=106
running=0
oncpu=1
values={Availability=1.0, oncpu=1, ResponseTime=81.0, number=107, ResponseCode=0.0, sleeping=106, running=0}
DEBUG [CollectorThread] [CollectorThread] CompletedTaskCount=0, ActiveCount=0, TaskCount=0, PoolSize=0
procmon Processes Availability:
exec:file=/tmp/proc.sh:Availability
Right, this is a way it works. Go to the bundle base directory where you find pdk dir which contains libraries for agent. Here's the parameters:
-p plugin - to use
-m want - dryrun to collect metrics
-Dplugins.include - load only given plugin
-Dlog - enable debugging
-t - what service to use (this can be checked from hq ui, usually server name + service name)

This is a command I used:
java -jar pdk/lib/hq-product.jar -p proc -m metric -Dplugins.include=proc -Dlog=DEBUG -t "procmon Processes"

Running this against your xml descriptor, there is a error which says:
-clip-
procmon Processes Availability:
exec:file=%script%:Availability
getValue failed for metric: procmon Processes:exec:file=%script%:Availability
IllegalArgumentException: File '%script%' does not exist
hit enter to continue
-clip-

Right, this means that script parameter is not visible to agent. 'script' parameter is actually coming from config values if defined like this. Now we can see that your config setting don't contain anything, we see the reason for failed metric collecting.

Then I added one config line(I have bash script for testing your values):
<service name="Processes">
<config>
<option name="script" description="script file" default="/tmp/proc.sh"/>
</config>

After this I run dryrun command again and output is:
-clip-
INFO [main] [SQLQueryMeasurementPlugin] Registered proxy for domain: sql
INFO [main] [ProductPluginManager] Loading plugin: proc-plugin.xml
DEBUG [main] [Collector] Adding exec collector: [/tmp/proc.sh]
INFO [main] [CollectorThread] CollectorThread started
DEBUG [CollectorThread] [CollectorThread] Created ThreadPoolExecutor: corePoolSize=30, maxPoolSize=30
DEBUG [CollectorThread] [Collector] Running exec collectors
DEBUG [CollectorThread] [ExecutableProcess] running: [/tmp/proc.sh]
DEBUG [CollectorThread] [Execute] Waiting process
DEBUG [CollectorThread] [Execute] End waiting, stop threads
DEBUG [CollectorThread] [Execute] Watchdog stopped
DEBUG [CollectorThread] [Execute] Stream handler stopped
DEBUG [CollectorThread] [Execute] Done exit=0 /tmp/proc.sh
DEBUG [CollectorThread] [ExecutableProcess] [/tmp/proc.sh]: Availability=0
number=107
sleeping=106
running=0
oncpu=1

DEBUG [CollectorThread] [Collector] name=exec, thread=CollectorThread, result=Fri Jun 12 09:40:24 EEST 2009 [Info] (/tmp/proc.sh) Availability=0
number=107
sleeping=106
running=0
oncpu=1
values={Availability=1.0, oncpu=1, ResponseTime=81.0, number=107, ResponseCode=0.0, sleeping=106, running=0}
DEBUG [CollectorThread] [CollectorThread] CompletedTaskCount=0, ActiveCount=0, TaskCount=0, PoolSize=0
procmon Processes Availability:
exec:file=/tmp/proc.sh:Availability
=>100.0%
procmon Processes Number of Processes:
exec:file=/tmp/proc.sh:number
=>107.0
procmon Processes Number of Sleeping Processes:
exec:file=/tmp/proc.sh:sleeping
=>106.0
procmon Processes Number of Running Processes:
exec:file=/tmp/proc.sh:running
=>0.0
procmon Processes Number of Processes on the CPU:
exec:file=/tmp/proc.sh:oncpu
=>1.0
INFO [main] [CollectorThread] CollectorThread stopped
-clip-

So I guess it will start working after you add that config option.

[tegulizard13]

Thank you, that portion worked just as you said, and now that test shows that metrics can be gathered.

In the HQ web interface when I add the server and the service, I still cannot see that any metrics are gethered (with graphs). I have already made sure that there is a metric interval set through the administration panel.

Is this a symptom of my xml descriptor or a problem with my configuration through the web interface?

For the server and the service, there is also a yellow bar along the top saying "This resource has not been configured. Please set its Configuration Properties." When I click "Configuration Properties" the next screen has another yellow bar stating "No plugin is available for this resource." and there is nothing to configure.

[jvalkeal_hyperi]

Well, strange.. it worked for me.

Are you looking correct place? So you have manually added server and service under that. Server don't have any metrics, service instead has your metrics.

Are you actually able to see those metrics in UI (under service) if you click 'show all metrics'? Then you should see all metrics even if hq is not collecting.

[tegulizard13]

I can't seem to get metrics to show up anywhere for this.

[jvalkeal_hyperi]

You need to click that 'please set its configuration properties'. It will open config window. There you should have parameter where you can set path to your script(the one you added earlier).

Thing is that on some cases, hq will not start collecting metrics unless you once go to configuration and click OK. This is usually case when plugin doesn't contain autoinventory methods.

[tegulizard13]

This is all I'm given for the "configuration properties" for both the server and the service.

[jvalkeal_hyperi]

Right, post your xml file!

Also make sure that both files on server and agent are identical. Now it seems that server doesn't know that there is a configuration parameter for your script path. I know that you added that config param if you were able to fetch metrics using dry-run.

[tegulizard13]

<plugin>
<server name="procmon">
<filter name="script" value="/erp/usr/local/hyperic/hyperic-hq-agent-4.0.3/bundles/agent-4.0.3-953/pdk/scripts/proc.pl"/>

<config>
</config>

<property name="BACKGROUND_COMMAND" value="false" />
<plugin type="measurement" class="org.hyperic.hq.product.MeasurementPlugin"/>

<service name="Processes">
<config>
<option name="script" description="script file" default="/erp/usr/local/hyperic/hyperic-hq-agent-4.0.3/bundles/agent-4.0.3-953/pdk/scripts/proc.pl"/>
</config>

<filter name="template"
value="exec:file=%script%"/>

<metric name="Availability"
template="${template}:Availability"
indicator="true"/>

<metric name="Number of Processes"
category="UTILIZATION"
indicator="true"
template="${template}:number"/>

<metric name="Number of Sleeping Processes"
category="UTILIZATION"
indicator="true"
template="${template}:sleeping"/>

<metric name="Number of Running Processes"
category="UTILIZATION"
indicator="true"
template="${template}:running"/>

<metric name="Number of Processes on the CPU"
category="UTILIZATION"
indicator="true"
template="${template}:oncpu"/>
</service>
</server>
</plugin>

It works now, I had forgot to update the file sitting on the server.
I'm not sure I need this up at the top:
<filter name="script" value="/erp/usr/local/hyperic/hyperic-hq-agent-4.0.3/bundles/agent-4.0.3-953/pdk/scripts/proc.pl"/>
But I'll find out.

Thank you so much for your help. Now we will be able to fully migrate over to Hyperic.

[jvalkeal_hyperi]

Nice that you got it :) Right, few words about filters/templates.

On general level you don't need to use filters. It's only a way to simplify xml structure. You are now setting template in metric as template="${template}:Availability". This template information is always something what metric collector needs to know. However, if you set global template(outside/before metric tag) and it's name is 'template', its automatically picked up by metric.

You could also write your global template filter like this:
<filter name="template" value="exec:file=%script%:${alias}"/>

In this case your metric definition doesn't need to have template attribute at all. Now you have it, since you are adding alias manually. Lets new look few metric definitions. If using above filter, availability tag could be written as:
<metric
name="Availability"
indicator="true"/>

or
<metric
name="Availability"
alias="Availability"
indicator="true"/>

You noticed that first definition doesn't contain alias attribute. If template needs alias, there is an extra check in hq regarding this issue. If alias is missing, it's automatically calculated from metric name attribute(by removing possible white spaces).

Next example:
<metric name="Number of Processes"
category="UTILIZATION"
indicator="true"
template="${template}:number"/>

This could be written as:
<metric
name="Number of Processes"
alias="number"
category="UTILIZATION"
indicator="true"/>

or

<metric
name="number"
category="UTILIZATION"
indicator="true"/>

or ( in case your script would return "NumberofProcesses=34"

<metric
name="Number of Processes"
category="UTILIZATION"
indicator="true"/>

[tegulizard13]

accidental double post, my apologies

Message was edited by: tegulizard13

[tegulizard13]

Relevant log info:

2009-06-11 15:17:45,807 INFO [Thread-0] [ProductPluginManager] Loading plugin: proc-plugin.xml
2009-06-11 15:17:45,841 INFO [Thread-0] [AgentDaemon] Product Plugin Manager initalized
2009-06-11 15:17:45,847 INFO [main] [AgentConnection] 127.0.0.1:2144 -> agent:ping
2009-06-11 15:17:45,852 INFO [Thread-0] [AgentCommandsServer] Registering Agent Commands Service with Agent Transport
2009-06-11 15:17:45,852 INFO [Thread-0] [AgentCommandsServer] Agent commands started up
2009-06-11 15:17:45,888 INFO [Thread-0] [AutoinventoryCommandsServer] Registering AI Commands Service with Agent Transport
2009-06-11 15:17:45,891 INFO [Thread-0] [AutoinventoryCommandsServer] Autoinventory Commands Server started up
2009-06-11 15:17:45,919 INFO [Thread-0] [CommandsServer] Commands Server started up
2009-06-11 15:17:45,933 INFO [Thread-0] [ControlCommandsServer] Registering Control Commands Service with Agent Transport
2009-06-11 15:17:45,933 INFO [Thread-0] [ControlCommandsServer] Control Commands Server started up
2009-06-11 15:17:45,992 INFO [Thread-0] [SenderThread] Maximum metric batch size set to 500
2009-06-11 15:17:46,121 INFO [Thread-0] [TrackerThread] Event report batch size set to 100
2009-06-11 15:17:46,137 INFO [Thread-0] [MeasurementCommandsServer] Registering Measurement Commands Service with Agent Transport
2009-06-11 15:17:46,302 INFO [Thread-0] [MeasurementCommandsServer] Measurement Commands Server started up
2009-06-11 15:17:46,311 INFO [Thread-0] [LiveDataCommandsServer] Registering Live Data Commands Service with Agent Transport
2009-06-11 15:17:46,311 INFO [Thread-0] [LiveDataCommandsServer] Live Data Commands Server started up
2009-06-11 15:17:46,312 INFO [Thread-0] [AgentTransportLifecycleImpl] Agent is not using new transport.
2009-06-11 15:17:46,357 INFO [Thread-2] [ConfigPopulateThread] Starting config populate thread
2009-06-11 15:17:46,515 INFO [Thread-0] [AgentDaemon] Agent started successfully
2009-06-11 15:17:46,516 INFO [WrapperStartStopAppMain] [AgentConnection] 127.0.0.1:2144 -> agent:ping
2009-06-11 15:17:46,857 INFO [main] [AgentConnection] 127.0.0.1:2144 -> agent:ping
2009-06-11 15:17:47,141 INFO [WrapperStartStopAppMain] [AgentConnection] 127.0.0.1:2144 -> bizapp:getCAMServer
2009-06-11 15:17:47,155 INFO [Thread-2] [ConfigPopulateThread] Received measurement configuration for Solaris '1:10025'
2009-06-11 15:17:47,156 INFO [Thread-2] [ConfigPopulateThread] Creating config track plugin 1:10025
2009-06-11 15:17:47,158 INFO [Thread-2] [ConfigTrackPlugin] init file watcher for: 1:10025
2009-06-11 15:17:47,163 INFO [Thread-2] [ConfigPopulateThread] Creating log track plugin 1:10025
2009-06-11 15:17:47,183 INFO [Thread-2] [RunnableTrackThread] RunnableLogThread started
2009-06-11 15:17:47,267 INFO [ScheduleThread] [CollectorThread] CollectorThread started
2009-06-11 15:17:47,806 INFO [main] [AgentConnection] 127.0.0.1:2144 -> bizapp:getCAMServer

Still not working, I'll try what you said most recently now.