DX Unified Infrastructure Management

 View Only
  • 1.  Controlling when to send clears with custom probes

    Posted Oct 03, 2012 07:19 PM

    Community,

    I am developing a custom probe to check a db and do some logic per row and create an alarm if the criteria is met. I understand how to check the db, create an alarm, send QOS. What I am unclear on is how the Clearing process works. I know i can send a clear and include the suppression string info and it will clear it but how do I keep up with what has an alarm currently opened to know if I need to send a clear. If im looping thousand's + rows and checking if row is now > 0 then alarm it seems if that row is 0 and i send a clear on every interval the probe runs the check that would be alot of non needed clear messages flooding the NIM-BUS. How do you guys in know if there is an alarm currently active to know if a clear needs to be sent? My language using is VBScript.

     

     

    Thanks,

    Jarrod



  • 2.  Re: Controlling when to send clears with custom probes

    Posted Oct 03, 2012 07:37 PM

    What I have typically done is built a data structure that would hold alarm status information. In Perl this is a hash, and in Lua it is a table. The keys in the data structure are the alarm suppression keys. Based on what I find in the data structure, I can determine which of the following is true and take the appropriate action when the check is clear:

     

    • Previous check resulted in a non-clear alarm message - send a clear
    • Previous check was clear - send nothing
    • There was no previous check - send a clear

    I think VBScript has a corresponding data structure that you could use similarly. This should work if you run the probe as a daemon. I know the VBScript SDK does not work very well for daemon probes (compared to the other SDKs), but you can probably still run it as a daemon and just have it sleep rather than relying on the SDK to create an event loop. If you are running the probe on interval rather than as a daemon, this solution will not work very well. You would need a way to save the state of the data structure to disk before exiting each run.



  • 3.  Re: Controlling when to send clears with custom probes

    Posted Nov 29, 2012 09:03 PM

    Keith,

    Would you happen to have any code i could use to re-engineer for creating something similar to what I am looking at doing? Language doesnt matter, whatever is cleanest i will use.



  • 4.  Re: Controlling when to send clears with custom probes

    Posted Nov 29, 2012 09:31 PM

    Here are some snippets that might be useful...

     

    Just to give you some context, this is from a probe written in Lua that simply attempts to contact each robot to make sure it is reachable. It counts the number of failures per robot and only sends an alarm when the failures exceed a specific threshold (the 'failures' variable below, which I believe I have set to 3).

     

    Creating a table to track alarms:

     

    local alarms  = {}

     Doing the check (which occurs inside a loop):

     

    local suppkey = probename.."/"..robot
    
    if rc == NIME_OK then
        if alarms[suppkey] == nil or alarms[suppkey] >= failures then
            probe.log(1, "Request to robot %s successful, sending clear", nimaddr)
            nimbus.alarm(NIML_CLEAR, "Robot "..nimaddr.." is reachable", suppkey, nil, robot)
            alarms[suppkey] = 0
        else
            probe.log(2, "Request to robot %s successful", nimaddr)
        end
    else
        if alarms[suppkey] == nil then
            alarms[suppkey] = 1
        else
            alarms[suppkey] = alarms[suppkey] + 1
        end
        if alarms[suppkey] >= failures then
            probe.log(1, "Request to robot %s failed %d times (error = %d), sending alarm", nimaddr, alarms[suppkey], rc)
            nimbus.alarm(alarm_level, "Robot "..nimaddr.." is NOT reachable", suppkey, nil, robot)
        else
            probe.log(1, "Request to robot %s failed %d times (error = %d)", nimaddr, alarms[suppkey], rc)
        end
    end