DX Infrastructure Manager

Expand all | Collapse all

Monitoring with logmon

Jump to Best Answer
  • 1.  Monitoring with logmon

    Posted 2 days ago
    Hi everyone.
    I need monitoring a port in state "listen" locally.
    I can't to use net_connect because isn't permit make query to the network for security reason of customer. So my estrategy focus in to use logmon probe.

    This is my configuration:
    1) I am capturing the state the port with a profile in the command mode. (netstat -n -a | grep tcp | grep "LISTEN" | grep -v "grep" > /opt/nimsoft/probes/system/logmon/netstat.log). The output of the profile is ---->

    tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
    tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:48000 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:48001 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:48007 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:48008 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:48009 0.0.0.0:* LISTEN
    tcp 0 0 0.0.0.0:48010 0.0.0.0:* LISTEN
    tcp6 0 0 :::22 :::* LISTEN
    tcp6 0 0 ::1:25 :::* LISTEN

    2) I am reading the log (/opt/nimsoft/probes/system/logmon/netstat.log), for example with this regex: /127.0.0.1:25\s+0.0.0.0:\W\s+LISTEN/
    This is regex working god, and generate an alarm ---->


    But although the above is fine, I need the opposite, that the alert is generated when (127.0.0.1:25 0.0.0.0:* LISTEN) is not found, which would indicate that the port is no longer connecting up.

    Is possible this with logmon? Or maybe I need to change my regex? I don't know how to meet this requirement

    Any idea?


  • 2.  RE: Monitoring with logmon
    Best Answer

    Posted 2 days ago
    HI Miller, You could use negative RegX and you could use command mode or you could use exit code monitoring in command mode. If you run your program in command load(don't read file) netstat -n -a | grep (your pattern you want to see). If it finds the exit code will be 0 if not 1. See example. You could also | to wc -l and use a variable for operend (<,>,=), No need to create file.

    [root@REDHATTest ~]# ls -l
    total 56
    -rw-------. 1 root root 6921 Jan 28 2019 anaconda-ks.cfg
    -rw-------. 1 root root 6577 Jan 28 2019 original-ks.cfg
    -r-xr-xr-x. 1 root root 2102 Jul 5 2019 rc.local
    -rw-r--r--. 1 root root 3 Jun 17 13:18 sepap-install.log
    -rw-r--r--. 1 root root 267 Feb 29 19:20 sepfl-install.log
    -rw-r--r--. 1 root root 11010 Mar 18 2019 sepfl-install.log.13372
    -rw-r--r--. 1 root root 267 Jul 2 2019 sepfl-install.log.15773
    -rw-r--r--. 1 root root 267 May 22 2019 sepfl-install.log.16277
    -rw-r--r--. 1 root root 243 Mar 18 2019 sepfl-kbuild.log
    -rw-r--r--. 1 root root 629 Mar 18 2019 sep-install.log
    [root@REDHATTest ~]# echo $?
    0
    [root@REDHATTest ~]# ls -l |grep GG
    [root@REDHATTest ~]# echo $?
    1
    [root@REDHATTest ~]# ls -l |grep log
    -rw-r--r--. 1 root root 3 Jun 17 13:18 sepap-install.log
    -rw-r--r--. 1 root root 267 Feb 29 19:20 sepfl-install.log
    -rw-r--r--. 1 root root 11010 Mar 18 2019 sepfl-install.log.13372
    -rw-r--r--. 1 root root 267 Jul 2 2019 sepfl-install.log.15773
    -rw-r--r--. 1 root root 267 May 22 2019 sepfl-install.log.16277
    -rw-r--r--. 1 root root 243 Mar 18 2019 sepfl-kbuild.log
    -rw-r--r--. 1 root root 629 Mar 18 2019 sep-install.log
    [root@REDHATTest ~]# echo $?
    0




  • 3.  RE: Monitoring with logmon

    Posted 2 days ago
    Oh ... Gregg, I didn't know that logmon could work this way.
    I configured the command monitoring like this ---->

    Exist some way of clear the alarm when the condition change?



  • 4.  RE: Monitoring with logmon

    Posted 2 days ago
    You can get rid of the grep -v grep. The exit code only applies to the last grep/command.

    Clearing is a little tricky. I think I have successfully used this doc to do it.

    https://knowledge.broadcom.com/external/article?articleId=135865


  • 5.  RE: Monitoring with logmon

    Posted 2 days ago
    Or to clear you can just add another setting in your threshold window of the exit code settings, to send a clear on a 0....I think so anyway. Try it out and post if that worked for the clear.


  • 6.  RE: Monitoring with logmon

    Posted 2 days ago
    Edited by Miller Grisepe Echagarreta Parra 2 days ago
    Gregg, I thought on the solution that you explain, but the severity "clear" not exist. Maybe this is an improvement.



    The solution that I thought is to create a new AO in NAS that find the alarm of logmon with expression filter and "Close" action, on overdue age = 4s. In my LAB it's working.


  • 7.  RE: Monitoring with logmon

    Posted 2 days ago
    Yes that would be too easy, maybe an enhancement. You could get a clear if the command you used was netstat -an | grep (pattern to watch) | wc -l. The output would be 1 from wc -l if it finds 1 match and 0 if no match. Then you could create 2 watchers which would be test your output in the variable section on alarm according to value.

    The "Ping section" in this doc will help. In your case the "Match Expression" would be just a "*" in your 2 watchers but your varible section would use operend to check for your command value.
    https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/it-operations-management/ca-unified-infrastructure-management-probes/GA/alphabetical-probe-articles/logmon-log-monitoring/logmon-use-case-examples.html


  • 8.  RE: Monitoring with logmon

    Posted 2 days ago
    You can post the output through "wc -l" to count the lines returned - then you can have watchers for "/^0$/" to get the error and "^[^0].*" to get everything else to clear.


  • 9.  RE: Monitoring with logmon

    Posted 2 days ago
    Excellent Gregg, this solutions it's much better. This is my command.

    netstat -an | grep "LISTEN" | grep 127.0.0.1:25 | grep 0.0.0.0:* | wc -l

    My whatchers are ----> /0/ = port down and /1/ = port establlished

    Thanks so much for teach me.


  • 10.  RE: Monitoring with logmon

    Posted 2 days ago
    No problem Miller, and you are using Garin's suggestion for evaluation, I do it in the variables tab....good luck.


  • 11.  RE: Monitoring with logmon

    Posted yesterday
    One important point when it comes to the sending of a "clear" message from logmon (actually important when using logmon in general) is that you should set the suppression key for each watcher rule and not just let it be the logmon default. If you have the two watcher rules (one that sends an alarm and one to clear) the suppression key for both rules should be identical. Even when not sending clear messages, the suppression key should be set for each watcher rule to ensure each alarm is separate and distinct from other alarms coming from other profiles and watchers or one profile with multiple watchers configured in the same logmon probe instance. You can use built-in variables so that the key contains the profile and watcher names.

    ------------------------------
    [Designation]
    [City]
    ------------------------------



  • 12.  RE: Monitoring with logmon

    Posted 2 hours ago
    Thanks James, your information it's true. In fact Gregg published this link https://knowledge.broadcom.com/external/article?articleId=135865