Probe failure

[B.Nikhil]

Hi,

 

 I need a help .  Is there any way to check security validation on probes and bring it back operational using callback without generating alerts ?

 

 

Thanks,

Nikhil

[hitesh sehgal]

A possible workaround can be identifying such alerts using nas auto operator and trigger a lua script to validate the probes.

 

this article will help you out:

 

Automatically validate hdb and spooler probes via - CA Knowledge 

[B.Nikhil]

Thanks Hitesh. I have this script, but needs to know if this script  automatically activate  the probes or not. Say for instance, due to firewall blocking , if this alert generates ,will it automatically activate the probes or wait until the firewall port is opened ?

[hitesh sehgal]

Well, port blocking needs to be fixed as per my knowledge so that script ran by NAS probe can reach to the controller and call the callbacks.

 

This script has 2 callbacks one for probe_verify and second for probe_activate.

 

In your environment, are such alerts always have port blocking in addition to ip change/reimage or cloning.

[B.Nikhil]

Yes, not frequent , but has seen in some instances. Is there any way we can know that this is due to port blocking or not ? Any notification mechanism ?

[hitesh sehgal]

Not sure, but maybe you can modify the same script and add a check on telnet to the server on port 48000 and if failing trigger another alert in script itself. But in my opinion, it will cause more alerts and overhead in addition to probe failed to start alerts.

[B.Nikhil]

I have applied the script, but the strange thing is that probes are not validated, still sitting in red colour, but alerts are not generated. Any idea what could be the reason ?

[B.Nikhil]

Will it work for already generated alerts ? I am not seeing activating the probe.

[hitesh sehgal]

It depends on how you set up the auto operator profile. try setting it up on every interval and see the profile activity, it will show at what time your specific profile ran.

 

 

For more information, just go through the documentation once

The nas Auto-Operator Tab - CA Unified Infrastructure Management Probes - CA Technologies Documentation 

[B.Nikhil]

I set it up as 'on arrival' . Let me try the option 'On every Interval'

[Garin Walsh]

There's some folklore around how well "On Arrival" actually works. Some of it would argue that, depending on the version of nas you are running, you don't want to use this.

 

On arrival will fire your script every time a matching message arrives. This may not be what you want especially where some probes send a message every time they test - you can wind up running a script many times unexpectedly that way.

 

On Interval just tests if the condition for the script is true on each AO interval. Kind of like on arrival except it's on timer.

 

What you probably want is "on overdue age". This runs the script one time once the alarm has existed for at least the specified length of time. So if you created your profile with an overdue age of 5 minutes, when you saved it, the nas would have gone through the list of events that matched the profile's criteria and scheduled execution of the script for creation time + 5 minutes. Anything in the past immediately satisfies the criteria to run and so does.

 

The nice thing about this is that if the alert only exists for 3 minutes, the script never fires.

 

-Garin

[B.Nikhil]

Thanks Grain. Why i chose 'on arrival' is that , we have time interval 3min set on NAS rule on Primary hub to assign any major alerts to a user account ( Remedy Ticketing system). So if i select for overdue , this alert will get assigned to the Remedy user account. We are looking to avoid it before reaching the 3min threshold.

[B.Nikhil]

I tried it , no alerts generated other than  "Specified IP change " one. Is there any way we know the  script has ran ? Show activity logs from the script is not showing any information and it is blank when selected last 50 and by date.

[Britta Hoffner]

Hello Nikhil, is your question answered or do you still need assistance for this ?

 

Kind regards,

Britta Hoffner

CA Support

[B.Nikhil]

Let me check if 'on every interval' will works or not in my environment.

[B.Nikhil]

The details of security validation is writing on NAS log using printf statement.Is there any code which you know can be added to print to any file or as separate alert?

[hitesh sehgal]

you can use logmon probe to monitor nas logs and match the relevant details using regular expressions and can trigger an alert.

[B.Nikhil]

Do you know the  regular expression to get the probe address from nas.log to trigger the alert?

[hitesh sehgal]

can you paste a snaphshot from the log file and what you are exactly trying to capture.

[B.Nikhil]

 

New Alert should be Probe Successfully Validated  followed by Probe address

[hitesh sehgal]

simply create a logmon profile on nas.txt log file.

create a watcher, set it to updates mode and in the expression to match put something like "*probe successfully validated*". set the message severity level and do not put anything in message to send and it will populate the complete line in the message text.

 

there are a lot of good articles for logmon in the community, try doing this and let us know the results.

[B.Nikhil]

Another interesting issue found is that logmon probe is not  activating after this script ran. All other probes got activated. I need to manually restart the robot to bring it online and activate. Is there any way we can include robot restart in the script to resolve this issue ?

[B.Nikhil]

It worked . Thanks Hitesh

[Garin Walsh]

logmon is dependant on hdb running first. Therefore order of events is important. I'd suggest enhancing the script to validate and activate hdb first (and wait a short period) if the probe triggering it is logmon.

 

-Garin

[B.Nikhil]

Thanks Garin for the information. I tried the following robot reboot script , when logmon throws error " Probe Logmon failure"

nimbus.request("/Domain/Hub/Robot/controller", "_stop")

But it is not working as expected. Please let me know if any changes needed in the  script.

[David Michel]

by default controller.cfg should be set to start logmon after hdb.

example:

<logmon>
   description = Log monitor
   group = System
   active = yes
   type = daemon
   command = logmon.exe
   config = logmon.cfg
   datafile = logmon.dta
   logfile = logmon.log
   workdir = probes/system/logmon
   start_after = hdb