DX Infrastructure Manager

Expand all | Collapse all

Issues with email gateway

  • 1.  Issues with email gateway

    Posted 01-02-2019 11:10 AM

    We are having issues where the email gateway is sending a clear message but we are not receiving the original alarm notification, even if the auto operator in the NAS has both the critical and clear boxes checked. This occurs with multiple different  probes, but became really apparent with a dirscan probe alert recently.  Has anyone else encountered this behavior? 

     

    TIA

    Happy New Year



  • 2.  Re: Issues with email gateway

    Posted 01-02-2019 12:03 PM

    Some probes will send clear alarms as part of their startup, could the probes have been restarted?

    Do you need to send an email on the clear alarm? If you do then could you use the alarm count >1 in the AO profile ?

    HTH

     



  • 3.  Re: Issues with email gateway

    Posted 01-02-2019 04:25 PM

    Don't know of a 'oh yeah it's this' quick fix, and recommend sending a test alarm and follow it through the process to find the break point.



  • 4.  Re: Issues with email gateway

    Posted 01-02-2019 06:32 PM

    pmurt, what are the AO's matching on? Can you provide a screenshot or example of the AO?

    From my experience, the content of a Critical alarm and a Clear alarm can be very different. I would only check both Critical and Clear if I'm 100% certain the content of both alarms are the same except severity. In my environment, I typically have a separate AO for the Clear alarm. This does mean double the number of AO's but my environment is quite complex and it's the only way to have notifications working the way we want.

     

    Also, whenever I'm troubleshooting these kind of problems, I use the Alarm History in the Status tab of the NAS probe. From there I can see the full transaction of the alarm and can view the content of Critical and Clear alarm to make sure the AO's match as required.



  • 5.  Re: Issues with email gateway

    Posted 01-03-2019 10:03 AM



  • 6.  Re: Issues with email gateway

    Posted 01-03-2019 10:09 AM

    I would recommend you change it from "on message arrival” to "on message overdue 1m” otherwise you will get an email on every interval of the alarm…. Unless that is what you want ?

    I presume you want one email when triggered and one when cleared ?



  • 7.  Re: Issues with email gateway

    Posted 01-03-2019 01:43 PM

    Also note that you are filtering on "count < 2" and so, if this is like most UIM probes, you'll get an increment to that count every test interval. It won't be <2 for very long.

     

    And Rowan is also right - Overdue age should be the default here. You will only want to select "on arrival" if you specifically know that you need to use that setting.

     

    And MaxL is also right - get in the habit of sealing with clear separate from the other alarm levels - it'll save you headaches in the future - especially because some probes don't send informative messages when they clear something, they depend on the supp key matching for that. 



  • 8.  Re: Issues with email gateway

    Posted 01-03-2019 06:05 PM

    pmurt, The suggestion to use "On overdue age" is very valid and required, depending on your requirement. If you need some kind of delay before the notifications are sent, then that is the option to use. Keep in mind though, If you have a lot of AO's you can end up with more than one AO triggering for each alarm. You would need to make sure for each AO, it excludes the filters of all other AO's.

     

    On the Advanced tab, there's an option to "Skip further profile checks on match". However, this option can only be selected when "On message arrival" is used. As Rowan mentioned, this typically causes you to get a notification for every interval count of the alarm. So to get around this, make sure "Message Counter" is "Equals to" value of "1".

     

    Then have a separate AO for the Clear alarm with the "Message Counter" set to "Great than or Equal to 1".

     

    So you end up with two AO's configured like this:

    AO1:

    Hostname = Servername

    Severity = Critical

    Message Counter = Equals to 1

    NMS Domain = mgmri

    Message String = /.*cashless*./

    Adv Tab = use "Skip further profile checks on match"

     

    AO2:

    Hostname = Servername

    Severity = Clear

    Message Counter = Greater than or Equal to 1

    NMS Domain = mgmri

    Message String = /.*cashless*./

    Adv Tab = use "Skip further profile checks on match"

     

    The only new consideration now when using the "Skip further profile checks on match" option, is the order of your AO's. They get processed top down and stop when there's a match. So have your AO's with more specific filter options at the top, and the more general ones down the bottom.



  • 9.  Re: Issues with email gateway

    Posted 01-07-2019 05:01 PM

    I have gone ahead and started ordering as you recommended and using the Skip further checks feature. A question we have about this. We are doing both Spectrum notifications and Email notifications. When ordering in NAS AO and using the 'Skip Further' does the system look at emails as a group and you just order the emails and group the spectrum alerts separately. Or do you order by probe, Emails and spectrum alerts as a whole?



  • 10.  Re: Issues with email gateway

    Posted 01-07-2019 09:58 PM

    I'm not familiar with Spectrum and not quite sure I fully understand your question. But in terms of how you order or group your AO's, there's no right or wrong way. It depends what works best for you. In my environment, I group the AO's by probe and then sort accordingly within that probe category. Actually, I use quite a few fields to filter on. Typically I use Origin, Probe Name, Sybsystem String, Sybsystem ID, User Tag 1, User Tag 2, Message string, and Hostname in each AO I have. The order in which I just mentioned them would also be typically be Generic to Specific if that makes sense. So basically I group by Probe type, then sort it so the more specific is at the top and less specific at the bottom. I also make sure the naming convention is consistent to make managing and administration easier. 

    E.g.

    AO-Disk1 (Origin, Probe, UserTag1/2, Message String, Hostname)

    AO-Disk2 (Origin, Probe, UserTag1/2, Message String)

    AO-Disk3 (Origin, Probe, UserTag1/2)

    AO-Disk4 (Origin, Probe)

    AO-CPU1 (Origin, Probe, UserTag1/2, Message String, Hostname)

    AO-CPU2 (Origin, Probe, UserTag1/2, Message String)

    AO-CPU3 (Origin, Probe, UserTag1/2)

    AO-CPU4 (Origin, Probe)

    AO-Service1 (Origin, Probe, UserTag1/2, Message String, Hostname)

    AO-Service2 (Origin, Probe, UserTag1/2, Message String)

    AO-Service3 (Origin, Probe, UserTag1/2)

    AO-Service4 (Origin, Probe)



  • 11.  Re: Issues with email gateway

    Posted 01-08-2019 10:25 AM

    The Spectrum AO's utilize a "repost" option as opposed to the "email"  in the action type.  We use utilize most of the actions, email, repost, command, assign, script, and close.