Is there a way to send a clearing alert (SNMP Trap) from APM v10.1 to SOI v3.4

[Manish Parikh]

Currently as it's architected...

 

When a "danger" threshold is breached in an Introscope APM alert there is a corresponding "action" associated. That "action" is to send an SNMP trap to SOI via SOI Connector. In parallel, there is also a "caution" alert defined with the same "action". When the alert threshold value goes below the "danger" value, it qualifies for breaching the "caution" threshold value. At which point another snmp trap "action" is sent to SOI. This acts as a clearing alarm/event/alert for SOI so that it can clear the danger alert associated with respective service.

 

If a "caution" alert "action" is not set (configured) and left empty.....there is no way for SOI to determine if that "danger" alert has subsided/reduced. In other words, there is NO clearing alert that's sent from APM if the danger threshold is subsided (data point has gone below the threshold value).

 

Is there a way for APM to send a clearing alert (or SNMP trap) to SOI so that when the "danger" alert is reduced, SOI can get the info and act accordingly?

 

Thanks in advance for anyone/everyone's assistance.

 

Jeff_Morris

 

Cheers,
Manish    

[Legacy User]

Manish,

 

This has to do with how you have the Alert configured in APM.

 

When using the SNMP Alert Action in conjunction with an alert, you need to choose “Whenever Severity Changes” from the Alert drop-down.  This tells APM to send the notification (SNMP trap in this case) when severity increases AND when it decreases.  This also includes transitions between states.   So, when the state of an alert goes from Danger to Caution, you will get a trap which will update SOI.  Then when it goes from Caution to Normal, SOI will clear the alert.

 

I believe this is the behavior you are looking for, and the way all APM alerts need to be configured when associated with the SNMP Alert Action.

 

Jeff

[Manish Parikh]

Jeff_Morris

Yeah, so what you explained is what I already have in place and remember being the criteria for SOI SNMP traps to work. But that's not what I was looking for.

 

If the alert "action" (sent to SOI as an SNMP trap) was ONLY associated with a "danger" alert and (and not with "caution") alert then what would happen? If the severity decreased from "danger" to "caution", would there be anything sent to SOI as a clearing alarm?

 

Manish

[Legacy User]

Manish,

 

If you ONLY have the action associated with Danger and not with Caution, here’s what would happen.

 

 

1)      Alert goes to danger state in APM

 

2)      APM sends notification to SOI which shows a Critical Alert

 

3)      Alert goes from danger state to caution in APM

 

4)      APM sends notification to SOI which updates severity to Major

 

5)      Alert goes from caution to normal state in APM

 

6)      NOTHING happens because you don’t have the Action configured on Caution. (i.e. you told APM not to notify)

 

You need to associate the Action with BOTH Caution and Danger thresholds in APM for the integration to work properly

 

Jeff

[Manish Parikh]

Jeff,

 

so on step # 3 & 4..........

Could SOI be configured in such a way that the severity changes from "Critical" to "Normal"?

 

App owners ONLY want to be notified on "Danger/Critical" alerts only. Could SOI service monitoring be configured so that it only tracks "Critical"?

 

Manish

[Legacy User]

Manish,

 

Is there a reason that you don’t want to configure the action on both state transitions?  I’m having a hard time understanding the downside to this approach

 

Jeff

[Manish Parikh]

Jeff,

With the current approach (as you have described) which is how it's designed in our environment. When alert goes from "danger" to "caution", it clears SOI alarm from "Critical" to "Major". At that point isn't an email alert for the "Major" alarm sent to the app owner? There is already an email being sent to the app owner when SOI receives "Critical" alarm. Now, with moving from "Critical" to "Major", does the app owner get another email notification?

Manish

[Hallett German]

Hi Manish:

Please let us know if Jeff's second response is more of what you are looking for or if you need additional assistance

Thanks

Hal German

[Manish Parikh]

Hal,

Please don't mark this as "answered" yet. It's not satisfactory to me yet.

 

Thanks

Manish

[Hallett German]

Thanks Manish. It seems that you are okay with the APM part of Jeff's response and now asking about the SOI alerting . (Which technically would need answering in another group.) We will both await Jeff_Morris  SOI part of the response.

Hal German

[Haruhiko Davis]

This whole issue has been discussed previously. There is even a KB article about this.

You must set Caution and Danger thresholds for SNMP traps to effectively "clear" an event.

[Manish Parikh]

Hiko,

Totally understand that and Jeff also explained it in the earlier thread. That's how it's designed in our environment.

 

Thanks
Manish

[Hallett German]

Hi Manish:

    I am trying to determine what questions that you need responses on. As far as I can tell, there are only two questions outstanding:

If the alert "action" (sent to SOI as an SNMP trap) was ONLY associated with a "danger" alert and (and not with "caution") alert then what would happen? If the severity decreased from "danger" to "caution", would there be anything sent to SOI as a clearing alarm?

 

 

Is that correct?

Thanks

Hal German

[Matthew Muskaloon]

If the alert "action" (sent to SOI as an SNMP trap) was ONLY associated with a "danger" alert and (and not with "caution") alert then what would happen?

You would only see the Danger alert in SOI.  If the trigger is "Whenever Severity Changes", then you would see the Danger cleared.

 

If the severity decreased from "danger" to "caution", would there be anything sent to SOI as a clearing alarm?

I believe that won't be sent to clear the alert if the trigger is set to anything other than "whenever severity changes". 

[Manish Parikh]

musma03

Ok, so if I understand it correctly, if my alert definition is as follows...

 

 

Then if the severity changes from "danger" to "caution"...........will SOI clear it's "Critical" alarm?

[Matthew Muskaloon]

Yes, the danger alarm will be cleared.  When it goes from caution to good, then that alarm will be cleared.

[Hallett German]

Hi Manish:

    I captured above what I believed are the two outstanding questions. Matt has responded on them.

   Please let us know any outstanding questions at this point so they can be answered. It would help from a readability standpoint to branch out the remaining questions and mark the previous ones as assumed answered or answered.

Thanks

Hal German

[Hallett German]

Hi Manish:

     Just checking in. What took place:

      - I posted what I saw as the only outstanding questions in this thread. Matt has responded to these with details and clarifications.

 

   It seems some good progress has been made on explaining alerts and how they are cleared.

   Please respond with any follow-up questions and outstanding questions that I may have missed from above to keep this moving.

Thanks

Hal German