Best way to create alerts based on a xml response from a URL

[devi ojha]

hello All , 

we have use case,
a XML response need to be parsed and alerts need to be created based on various elements in the xml .
Initially it was only one field to be checked so we used url_response to simply check for STRING_FOUND "<Status>Registered</Status>"​ and we we were fine :
sample alert was like "URL content for 'hostname-*****' contains the unexpected text '<Status>Registered</Status>'​ "​

however we now want to check for many common additional fields from the response . for example , in the below xml response we want to 
check for various <ConnectionStatus> of the microphones . So what would be the best way to parse the xml and create and alert like 
Microphone item="1" status Disconnected  ?

can we leverage restmon for this ? is it suitable for XML responses , if so can you please help me with a sample schema  ? or  is there any other better way to do this monitoring .

<Audio>

<Input>

<Connectors>

<HDMI item="1" maxOccurrence="n">

<Mute>On</Mute>

</HDMI>

<HDMI item="2" maxOccurrence="n">

<Mute>On</Mute>

</HDMI>

<Microphone item="1" maxOccurrence="n">

<ConnectionStatus>Connected</ConnectionStatus>

<Mute>Off</Mute>

</Microphone>

<Microphone item="2" maxOccurrence="n">

<ConnectionStatus>NotConnected</ConnectionStatus>

<Mute>Off</Mute>

</Microphone>

<Microphone item="3" maxOccurrence="n">

<ConnectionStatus>NotConnected</ConnectionStatus>

<Mute>Off</Mute>

</Microphone>

</Connectors>

<KeyClick>

<Attenuate>On</Attenuate>

<Detected>Silence</Detected>

<Enabled>Off</Enabled>

</KeyClick>

</Input>

<Microphones>

<Mute>Off</Mute>

</Microphones>

<Output>

<Connectors>

<Line item="1" maxOccurrence="n">

<ConnectionStatus>NotConnected</ConnectionStatus>

</Line>

</Connectors>

</Output>

<Volume>70</Volume>

<VolumeMute>Off</VolumeMute>

</Audio>

Thanks

------------------------------
Devi Ojha
------------------------------

[David Michel]

restmon is for monitoring devices or applications that respond to rest calls. 
Have you considered creating a different profile for each string to be monitored.

------------------------------
Support Engineer
Broadcom
------------------------------

[devi ojha]

Hello David , 

yes i was initially looking for creating a different profile for each string , but since the xml reponse from API will vary and there would be multiple entries of same string depending on number of input devices connected , like if the system is connected say with with input devices it will have two string with as 'Status'

Thanks

[Franklin Ravi D'souza]

Hi 

Can also explore logmon probe to parse the xml output 

https://docops.ca.com/ca-unified-infrastructure-management-probes/ga/en/alphabetical-probe-articles/logmon-log-monitoring/logmon-hints-and-examples#logmonHintsandExamples-Example3XMLFile

[Christian McHugh]

By convention many rest interfaces use json, but as there isn't a standard, it would be nice to have this assumption of the restmon probe documented. A rest interface can expose xml (as it does in this case) so it would be good to know if the restmon probe is only capable of walking json.

Using logmon to regex multiline xml may be the best option UIM is capable of, as it allows associating subkeys with the parent in order to monitor what is needed. However, if there are 30 items to be checks in that status.xml page, it sounds like this would require 30 logmon profiles, and thus 30 independent http calls to the device every polling interval. That doesn't sound very efficient. 

I guess the next best option is to write a custom python script to translate the needed bits, and to output to the logmon probe in a standard parsable format. Data could then be slurped into UIM as variables and acted upon. This seems to be the only workable solution for most monitoring of this nature (various custom interfaces), so is the UIM tool is mostly used as a framework to run custom scripts through the logmon probe -- in a nagios/sensu like fashion?

[Franklin Ravi D'souza]

Hi

Can also explore the logmon probe to parse xml file 

 https://docops.ca.com/ca-unified-infrastructure-management-probes/ga/en/alphabetical-probe-articles/logmon-log-monitoring/logmon-hints-and-examples#logmonHintsandExamples-Example3XMLFile

[devi ojha]

Thanks a lot @David Michel ,for responding ,different profile for each string is what we tried ​but thats not efficient as xml response would have same string multiple time across different input devices , like there might be 10 input devices and each would have entry for status .so its hard to parse .


@Frank, Thanks for that recommendation and example will try out logmon . 
​

------------------------------
Devi Ojha
MLP
------------------------------

[devi ojha]

Thanks a lot Frank , i will give a try with logmon probe .

[Justin Young]

Hi Devi Ojha

This XML looks very similar to XML returned from Tandberg Video Conferencing devices.

Turn the status into a metric that you can Alarm off in UIM using RESTmon would be very easy.

Could you give me an example of the URL you are calling that returned the XML at the top of this thread.

Regards
Justin

[Justin Young]

I created this quick xmlexample schema for RESTmon using the XML you provided and a guess at the url, I assumed the url likely uses Basic Authentication

RESTmon supports xpath to extract xml data the same as it uses jsonpath for json.

{
"xmlexample": {
"urls": [{
"id": "status",
"url": "/status.xml"
}],
"definition": {
"resource_category": null,
"defaults": {
"checkcert": "no",
"port": 443,
"interval": 60,
"httptimeout": 30000
},
"addedProfileFields": [{
"name": "username",
"value": "",
"type": "text",
"label": "Username"
},
{
"name": "password",
"value": "",
"type": "password",
"label": "Password"
}],
"auth": "basic",
"name": "xmlexample",
"type": "https"
},
"metrics": [{
"calculation": "",
"xml_ns": "",
"attributes": {
"uim": {
"qos_name": "QOS_HTTP_STATUS",
"ci_type": "2.2.2.2",
"qos_desc": "HTTP response status (State)",
"qos_abbr": "State",
"metric_type": "2",
"qos_unit": "State",
"source": "%hostname",
"target": "%urlid",
"defaultpublishing": "true"
}
},
"value": "%httpstatus",
"url": "%urlid",
"group": "Connections",
"helptext" : "This monitor indicates the following response codes: HTTP Status Response code 200 (OK),401 (Unauthorized) and 403 (Forbidden)"
},
{
"xml_ns": "",
"attributes": {
"uim": {
"conversion": "Connected:1, NotConnected:0, Default:-999",
"qos_name": "QOS_XMLEX_MICROPHONE_STATUS",
"metric_type": "9.9.9:1",
"qos_desc": "Microphone Status (Boolean)",
"qos_abbr": "boolean",
"qos_unit": "boolean",
"source": "%hostname",
"target": "/Audio/Input/Connectors/Microphone/@item",
"defaultpublishing": "true"
}
},
"value": "/Audio/Input/Connectors/Microphone/ConnectionStatus",
"url": "status",
"group": "General",
"helptext" : "Value 1 shows if microphone is connected and value 0 shows if it is not connected."
}],
"calculated_metrics": []
}
}

Hope this helps, let me know if you need anymore help

Justin

[devi ojha]

Thanks a lot Justin , That's  indeed the best solution.
yes it is similar to Tandberg Video Conferencing devices, we are trying to monitor CISCO Web-room Video conferencing devices using the cisco APIs. And this certainly helps a lot .

a few more questions 


1.  I see a Dashboard getting created on it own under operator console , however we don't have a yet cabi installed for 9.0.2 version hence we are not able to visualize . Where else we can see / populate these alarms/metrics ?  
2.  I am not able to see the metrics under metrics tab of configured host , attached a screen shot of same . is there any parameter/config need to  be enabled/configure to view the metrics 
3. I can see from logs , that its actually pulling the data and updating the database . Is there a way i can see the entries of database from UI ? or how do we see those details .

Logs : attached 
Thanks , 
Devi

------------------------------
Devi Ojha
------------------------------

Attachment(s)

losg.txt   5 KB 1 version

[Justin Young]

Hi Devi

I sent you a seperate message so we can get on a webex, so I can see the data going in work out where it would appear in USM and PRD.

Justin

[devi ojha]

Thanks a lot Justine , have responded you back in other thread , it would be great help indeed to come on webex to discuss lil more depth .

Thanks ,
Devi