DX Infrastructure Manager

Expand all | Collapse all

Probe failure

  • 1.  Probe failure

    Posted 06-27-2018 06:33 AM

    Hi,

     

     I need a help .  Is there any way to check security validation on probes and bring it back operational using callback without generating alerts ?

     

     

    Thanks,

    Nikhil



  • 2.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-27-2018 06:54 AM

    A possible workaround can be identifying such alerts using nas auto operator and trigger a lua script to validate the probes.

     

    this article will help you out:

     

    Automatically validate hdb and spooler probes via - CA Knowledge 



  • 3.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-27-2018 07:09 AM

    Thanks Hitesh. I have this script, but needs to know if this script  automatically activate  the probes or not. Say for instance, due to firewall blocking , if this alert generates ,will it automatically activate the probes or wait until the firewall port is opened ?



  • 4.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-27-2018 07:20 AM

    Well, port blocking needs to be fixed as per my knowledge so that script ran by NAS probe can reach to the controller and call the callbacks.

     

    This script has 2 callbacks one for probe_verify and second for probe_activate.

     

    In your environment, are such alerts always have port blocking in addition to ip change/reimage or cloning.



  • 5.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-27-2018 07:26 AM

    Yes, not frequent , but has seen in some instances. Is there any way we can know that this is due to port blocking or not ? Any notification mechanism ?



  • 6.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-28-2018 04:59 AM

    Not sure, but maybe you can modify the same script and add a check on telnet to the server on port 48000 and if failing trigger another alert in script itself. But in my opinion, it will cause more alerts and overhead in addition to probe failed to start alerts.



  • 7.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-29-2018 06:36 AM

    I have applied the script, but the strange thing is that probes are not validated, still sitting in red colour, but alerts are not generated. Any idea what could be the reason ?



  • 8.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-29-2018 07:08 AM

    Will it work for already generated alerts ? I am not seeing activating the probe.



  • 9.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-29-2018 08:41 AM

    It depends on how you set up the auto operator profile. try setting it up on every interval and see the profile activity, it will show at what time your specific profile ran.

     

     

    For more information, just go through the documentation once

    The nas Auto-Operator Tab - CA Unified Infrastructure Management Probes - CA Technologies Documentation 



  • 10.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-29-2018 09:35 AM

    I set it up as 'on arrival' . Let me try the option 'On every Interval'



  • 11.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-29-2018 10:23 AM

    There's some folklore around how well "On Arrival" actually works. Some of it would argue that, depending on the version of nas you are running, you don't want to use this.

     

    On arrival will fire your script every time a matching message arrives. This may not be what you want especially where some probes send a message every time they test - you can wind up running a script many times unexpectedly that way.

     

    On Interval just tests if the condition for the script is true on each AO interval. Kind of like on arrival except it's on timer.

     

    What you probably want is "on overdue age". This runs the script one time once the alarm has existed for at least the specified length of time. So if you created your profile with an overdue age of 5 minutes, when you saved it, the nas would have gone through the list of events that matched the profile's criteria and scheduled execution of the script for creation time + 5 minutes. Anything in the past immediately satisfies the criteria to run and so does.

     

    The nice thing about this is that if the alert only exists for 3 minutes, the script never fires.

     

    -Garin



  • 12.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-29-2018 10:36 AM

    Thanks Grain. Why i chose 'on arrival' is that , we have time interval 3min set on NAS rule on Primary hub to assign any major alerts to a user account ( Remedy Ticketing system). So if i select for overdue , this alert will get assigned to the Remedy user account. We are looking to avoid it before reaching the 3min threshold.



  • 13.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 07-02-2018 09:29 AM

    I tried it , no alerts generated other than  "Specified IP change " one. Is there any way we know the  script has ran ? Show activity logs from the script is not showing any information and it is blank when selected last 50 and by date.



  • 14.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Broadcom Employee
    Posted 06-29-2018 08:49 AM

    Hello Nikhil, is your question answered or do you still need assistance for this ?

     

    Kind regards,

    Britta Hoffner

    CA Support



  • 15.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 06-29-2018 09:36 AM

    Let me check if 'on every interval' will works or not in my environment.



  • 16.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 07-05-2018 10:33 AM

    The details of security validation is writing on NAS log using printf statement.Is there any code which you know can be added to print to any file or as separate alert?



  • 17.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 07-06-2018 08:05 AM

    you can use logmon probe to monitor nas logs and match the relevant details using regular expressions and can trigger an alert.



  • 18.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 07-06-2018 09:23 AM

    Do you know the  regular expression to get the probe address from nas.log to trigger the alert?



  • 19.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 07-06-2018 09:28 AM

    can you paste a snaphshot from the log file and what you are exactly trying to capture.



  • 20.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 07-06-2018 09:53 AM

     

    New Alert should be Probe Successfully Validated  followed by Probe address



  • 21.  Re: Probe '' FAILED to start, file check determines changes in the probe

    Posted 07-06-2018 10:25 AM

    simply create a logmon profile on nas.txt log file.

    create a watcher, set it to updates mode and in the expression to match put something like "*probe successfully validated*". set the message severity level and do not put anything in message to send and it will populate the complete line in the message text.

     

    there are a lot of good articles for logmon in the community, try doing this and let us know the results.



  • 22.  Re: Probe failure

    Posted 07-09-2018 10:05 AM

    Another interesting issue found is that logmon probe is not  activating after this script ran. All other probes got activated. I need to manually restart the robot to bring it online and activate. Is there any way we can include robot restart in the script to resolve this issue ?



  • 23.  Re: Probe failure

    Posted 07-10-2018 09:17 AM

    It worked . Thanks Hitesh



  • 24.  Re: Probe failure

    Posted 07-09-2018 11:25 AM

    logmon is dependant on hdb running first. Therefore order of events is important. I'd suggest enhancing the script to validate and activate hdb first (and wait a short period) if the probe triggering it is logmon.

     

    -Garin



  • 25.  Re: Probe failure

    Posted 07-10-2018 09:20 AM

    Thanks Garin for the information. I tried the following robot reboot script , when logmon throws error " Probe Logmon failure"

    nimbus.request("/Domain/Hub/Robot/controller", "_stop")

    But it is not working as expected. Please let me know if any changes needed in the  script.


  • 26.  Re: Probe failure

    Broadcom Employee
    Posted 07-11-2018 08:51 AM

    by default controller.cfg should be set to start logmon after hdb.

    example:

    <logmon>
       description = Log monitor
       group = System
       active = yes
       type = daemon
       command = logmon.exe
       config = logmon.cfg
       datafile = logmon.dta
       logfile = logmon.log
       workdir = probes/system/logmon
       start_after = hdb