DX Unified Infrastructure Management

 View Only
Expand all | Collapse all

Logmon to monitor latest file

  • 1.  Logmon to monitor latest file

    Posted Dec 10, 2019 02:51 AM
    Team,

    How do we monitor latest log file using logmon probe.

    Ex : we have application log file generating on daily basis with format "access_log.2019.12.05" , "access_log.2019.12.06",.."access_log.2019.12.09". 

    In the above example case, i would like to monitor the specific pattern in latest log file "access_log.2019.12.09". How to achieve this ?..

    I tried using access_log.%Y%m%d. but it doesnt work and gives me error "open file failed"


  • 2.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 05:19 AM
    You ask to check for a file: access_log.20191206


  • 3.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 08:57 AM
    Hi,

    I am able to fix it by using access_log.%Y.%m.%d and it works fine - fetches latest file.

    but now i am failing when i do test profile.. I created watcher rule, match expression *200* and when i tried with all type of file encoding, test result shows as "No result".

    File encoding type is in text/plain - charset US-ASCII. but we don't find these encoding type in logmon probe. How to test the profile with success result.

    Is there any way to add file encoding in logmon probe





  • 4.  RE: Logmon to monitor latest file

    Broadcom Employee
    Posted Dec 10, 2019 09:12 AM
    so to test the profile do not use wild cars in the file name.
    Pick one file to test with and make sure it works then move back to your date-time format. 
    this should allow you to test.

    On the General tab you can set the system encoding and the output encoding as needed
    http://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/it-operations-management/ca-unified-infrastructure-management-probes/GA/alphabetical-probe-articles/logmon-log-monitoring/logmon-im-configuration.html

    Update the following information to configure the character encodings of the log files in non english locales:
    •  
      Output Encoding: 
      defines the character encoding to generate alarms and QoS messages when the probe is deployed in a non english locale. The probe also uses the encoding to identify the file name that is specified in the monitoring profile.
    •  
      System Encoding: 
      specifies the encoding of the system where the probe is deployed.
    no other character set can be set in the probe at this time.


    ------------------------------
    Gene Howard
    Principal Support Engineer
    Broadcom
    ------------------------------



  • 5.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 09:39 AM
    Hi,

    I don't find the option output encoding / system encoding in general tab


  • 6.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 09:48 AM
    How did you code the "match expression" in the watcher?
    Try to use: /200/ (perl regex format)


  • 7.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 09:49 AM
    I tried with *200* - not working
    Now i tried with /200/ - Also not working


  • 8.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 09:57 AM
    Can you post your logmon.cfg, or the specific profile from logmon.cfg?


  • 9.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:06 AM






  • 10.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:06 AM
    Log file i am browsing is from Linux server


  • 11.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:16 AM
    I see a match: *ETC* in your watcher, what about 200?
    The goal of having the cfg file is that we can load that profile in our test system.


  • 12.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:28 AM
    Sorry i have sent wrong profile.. I have two profiles.

    Also i don't have access to the robot - hence not able to provide cfg file


  • 13.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:33 AM
    Hi,

    Attached the logmon.cfg file.


  • 14.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:35 AM
      |   view attached
    Request you to rename the file to cfg from txt.

    Attachment(s)

    txt
    logmon.txt   13 KB 1 version


  • 15.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:47 AM
    If I change the scanmode and file read position it works fine with me:
    scanmode = updates
    initialfileptr = 1
    resumefileptr = 4


  • 16.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:51 AM
    Hi,

    You tried with my logmon.cfg file ?.,

    how to set initialfileptr and resumeptr as 1 & 4


  • 17.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:56 AM
    Hi,

    After setting initial & resume ptr as 1 & 4 not working for me..

    Log file encoding is in us-ascii format.. Is this ok or anything to do with this


  • 18.  RE: Logmon to monitor latest file

    Broadcom Employee
    Posted Dec 10, 2019 11:26 AM
    so one thing to check.
    Can you vi the target file?
    If the target file is locked open you might be failing.
    scanfile = /appdata1/HTTPServer/logs/access_log.%Y.%m.%d
    or attach the lastest version of this for us to review.

    ------------------------------
    Gene Howard
    Principal Support Engineer
    Broadcom
    ------------------------------



  • 19.  RE: Logmon to monitor latest file

    Posted Dec 10, 2019 10:23 PM
    Hi,
    Will check and update..
    But when I try to browse and view file I am able to read it


  • 20.  RE: Logmon to monitor latest file

    Posted Dec 11, 2019 02:33 AM
      |   view attached
    Hi,

    Attached latest log file..

    Actual file name is access_log.2019.12.11. Since not able to upload the file as it is in format, changed the format to .txt. Request you to rename access_log.2019.12.11.txt to access_log.2019.12.11 and give a try. 

    Also the file encoding is us-ascii

    Attachment(s)

    txt
    access_log.2019.12.11.txt   575 B 1 version


  • 21.  RE: Logmon to monitor latest file

    Posted Dec 11, 2019 03:19 AM
    Your watcher with your file works fine with me, except that I execute it on a windows server (logmon version: 4.11)


  • 22.  RE: Logmon to monitor latest file

    Posted Dec 11, 2019 04:33 AM
      |   view attached
    Hi,

    Can you please check the attached file and update us.. Rename access_log.2019.12.10.txt to access_log.2019.12.10 and let me know the result.

    Attachment(s)

    txt
    access_log.2019.12.10.txt   271.51 MB 1 version


  • 23.  RE: Logmon to monitor latest file

    Posted Dec 11, 2019 05:31 AM
    I do not understand why you scan this file with mode: full (rescan the whole file) every 5 seconds where you will match 1000's lines?
    What is the goal here? With these settings you risk to overload logmon


  • 24.  RE: Logmon to monitor latest file

    Posted Dec 11, 2019 09:37 AM
    Hi,

    I am trying with mode: updates.. 

    For testing i kept it as 5 seconds.. We will change it to 10 mins later..

    Goal is to find the code ie *200* from the file and report an alert


  • 25.  RE: Logmon to monitor latest file

    Posted Dec 11, 2019 10:14 AM
    I would have a care with the way match expressions are specified. The documentation says:

    Constructing regular expression and pattern matching requires meta characters such as * and ?. The probe supports Perl Compatible Regular Expressions (PCRE) which are enclosed in forward slash (/). For example, the expression /[0-9A-C]/ matches any character in the range 0 to 9 in the target string.
    You can also use simple text with wild card operators for matching the target string. For example, the *test* expression matches the text 
    test in the target string.

    (note that the example cited above in the documentation is correct but incomplete as it leaves out that it also matches the letters A, B, and C.)

    I have never had consistent luck using the "simple text" wildcards. 

    You have:
          match = *20*

    which uses the simple wildcards. I would suggest it would be a much better habit to always use regex and rewrite your match as

        match = /.*20.*/ 

    This will eliminate any question of which matching methodology is in play and will save you from headaches when you have match patterns that include spaces and commas which are interpreted as separators in the simple method.

    There are also many resources on the Internet to help with writing and testing regex (regexpal.com for instance) - there is only the "test profile" option in the logmon GUI to verify the simple matching method.



  • 26.  RE: Logmon to monitor latest file

    Posted Dec 12, 2019 12:07 AM
    Hi,

    I tried /.*20.*/ and /[0-9A-C]/ - both doesn't work for me. Do i need to check anything on file encoding ?..


  • 27.  RE: Logmon to monitor latest file

    Posted Dec 12, 2019 02:49 AM
    With your file AND your config, even with scan mode: full it did work without any change.  BUT at that moment you are killing alarm_enrichment and your nas with the alarm ERROR that will be displayed in LOOP!!
    If you change in scan: update and set file read position to: End of file & Last read line, you will have to wait until a new line with the requested string is updated in the file.


  • 28.  RE: Logmon to monitor latest file

    Posted Dec 12, 2019 03:02 AM
    Hi Luc,

    Content in this file keep updating every seconds, hence there will always a new line will be keep written. but dono how it is failing in our environment


  • 29.  RE: Logmon to monitor latest file

    Posted Dec 12, 2019 03:04 AM

    Please find the scan mode and view file content



  • 30.  RE: Logmon to monitor latest file

    Posted Dec 12, 2019 03:09 AM
    Is your problem with alarm_enrichment solved? Perhaps is your logmon alarm in the queue?


  • 31.  RE: Logmon to monitor latest file

    Posted Dec 12, 2019 03:35 AM
    Hi,

    Alarm enrichment queue is on different HUB server.. 

    Logmon probe which i test is on different HUB server - its a separate PRI HUB server


  • 32.  RE: Logmon to monitor latest file

    Posted Dec 12, 2019 06:33 AM
    Hi All,

    HTTP/1.1 - - [12/Dec/2019:17:01:41 +0530] "POST /reqPayService HTTP/1.1" 202 -

    My goal is - just look for the number "202" in the above line and if matches trigger an alert


  • 33.  RE: Logmon to monitor latest file

    Broadcom Employee
    Posted Dec 12, 2019 07:32 AM
    I would use the following:
    /.*\s202\s-.*/

    ------------------------------
    Gene Howard
    Principal Support Engineer
    Broadcom
    ------------------------------



  • 34.  RE: Logmon to monitor latest file

    Posted Dec 12, 2019 07:48 AM
    Hi,

    I am getting below error in logmon log while scanning a file

    Dec 12 18:15:55:161 [140458863654656] logmon: unable to open data file logmon.dta


  • 35.  RE: Logmon to monitor latest file
    Best Answer

    Posted Dec 13, 2019 02:06 AM
    Hi,

    I am able to identify the problem and fix it.. HDB probe was missing in the robot, hence it was failing to read / test the pattern. After installing HDB probe, logmon probe works


  • 36.  RE: Logmon to monitor latest file

    Posted Dec 13, 2019 10:19 AM


    ------------------------------
    Daniel Blanco
    Enterprise Tools Team Architect
    DBlanco@alphaserveit.com
    ------------------------------