logmon probe max restarts

[Sid]

Logmon probe is going into max restart mode frequently.
When i deactivate/active probe and validate security it comes up but after some time it goes into that mode again and it continues. 
logmon 4.11
robt_update 7.96
DXIM 9.20

------------------------------
HCL Technologies
------------------------------

[David Michel]

This is very out of the ordinary. A fix via Security > Validate is normally a one time thing after a change has taken place to the robot, This might tie into your other thread about robot.cfg going blank. Do some research and find out what is making a change at these robots.

------------------------------
Support Engineer
Broadcom
------------------------------

[Sid]

Hi David,

robots are working ok for the servers where we have logmon probe with issues.
and those robit inactive and cfg blank issues are majorly on windows servers

------------------------------
HCL Technologies
------------------------------

[David Michel]

sure
consider capturing the problem in the logs, logmon at level 5, controller level 3.

------------------------------
Support Engineer
Broadcom
------------------------------

[Sid]

Jul 29 12:06:47:118 [140419014129408] Controller: Max. restarts reached for probe 'logmon' (command = logmon)
Jul 29 15:56:54:164 [139735612876576] logmon: ****************[ Starting ]****************
Jul 29 15:56:54:165 [139735612876576] logmon: * logmon 4.11
Jul 29 15:56:54:165 [139735612876576] logmon: * Nimsoft Corporation
Jul 29 15:56:54:165 [139735612876576] logmon: SREQUEST: probe_checkin ->10.134.135.42/48000
Jul 29 15:56:54:165 [139735612876576] logmon: head tout=15 addr=
Jul 29 15:56:54:165 [139735612876576] logmon: data type=1 nimcprid=5013541
Jul 29 15:56:54:165 [139735612876576] logmon: RREPLY: status=OK(0) <-10.134.135.42/48000 h=37 d=610
Jul 29 15:56:54:165 [139735612876576] logmon: head mtype=200 status=0 seq=0
Jul 29 15:56:54:165 [139735612876576] logmon: sockClose:0x129de00:10.134.135.42/24657
Jul 29 15:56:54:165 [139735612876576] logmon: SREQUEST: _close ->10.134.135.42/48000
Jul 29 15:56:54:165 [139735612876576] logmon: sslClientSetup - mode=0 cipher=DEFAULT
Jul 29 15:56:54:165 [139735612876576] logmon: nimSessionServer - port = 0
Jul 29 15:56:54:166 [139735612876576] logmon: sockCheckLocalPort: connect to port 48011 failed 111 (ok - its free)
Jul 29 15:56:54:166 [139735612876576] logmon: sockCheckLocalPort - close 8
Jul 29 15:56:54:166 [139735612876576] logmon: sockServer: next available port is 48011
Jul 29 15:56:54:166 [139735612876576] logmon: sockServer:0x129d9f0:0.0.0.0/48011
Jul 29 15:56:54:166 [139735612876576] logmon: * port=48011 PID=15984
Jul 29 15:56:54:166 [139735612876576] logmon: SREQUEST: port_register ->10.134.135.42/48000
Jul 29 15:56:54:166 [139735612876576] logmon: head tout=20 addr=
Jul 29 15:56:54:166 [139735612876576] logmon: data name=logmon port=48011
Jul 29 15:56:54:166 [139735612876576] logmon: RREPLY: status=OK(0) <-10.134.135.42/48000 h=37 d=0
Jul 29 15:56:54:166 [139735612876576] logmon: head mtype=200 status=0 seq=0
Jul 29 15:56:54:166 [139735612876576] logmon: sockClose:0x129fcc0:10.134.135.42/24661
Jul 29 15:56:54:166 [139735612876576] logmon: SREQUEST: _close ->10.134.135.42/48000
Jul 29 15:56:55:167 [139735612876576] logmon: (nanny) - child pid 15985 has exited with code 0
Jul 29 15:56:59:182 [139735612876576] logmon: fileFind - Directory: '/opt/eai/is/optimiser_be_nl/IntegrationServer/instances/default/logs/' Pattern: 'server.log'
Jul 29 15:56:59:182 [139735612876576] logmon: fileFind - getting file lock
Jul 29 15:56:59:182 [139735612876576] logmon: fileFind - got file lock
Jul 29 15:56:59:182 [139735612876576] logmon: fileFind - accept and save path '/opt/eai/is/optimiser_be_nl/IntegrationServer/instances/default/logs/'
Jul 29 15:56:59:182 [139735612876576] logmon: fileFind - Start file search
Jul 29 15:56:59:182 [139735612876576] logmon: fileFind - save file with path '/opt/eai/is/optimiser_be_nl/IntegrationServer/instances/default/logs/' and name 'server.log'
Jul 29 15:56:59:182 [139735612876576] logmon: fileFind - Number of input files: 1
Jul 29 15:56:59:182 [139735612876576] logmon: listFile - releasing file lock
Jul 29 15:56:59:182 [139735612876576] logmon: listFile - released file lock
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - found /opt/eai/is/optimiser_be_nl/IntegrationServer/instances/default/logs/server.log
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - adding /opt/eai/is/optimiser_be_nl/IntegrationServer/instances/default/logs/server.log
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - profile [Profile1] allocated
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - filename added: /opt/eai/is/optimiser_be_nl/IntegrationServer/instances/default/logs/server.log
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - added subject, alarm and qos
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - ident=name
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - added mode and interval
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - added formats
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - added max alarm message
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - added watchers
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - added exclude
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - added expansion
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - ending scan
Jul 29 15:56:59:182 [139735612876576] logmon: check_profiles - done with file search of /opt/eai/is/optimiser_be_nl/IntegrationServer/instances/default/logs/
Jul 29 15:56:59:789 [139734388762368] logmon: lgm: check format start..[0]
Jul 29 15:56:59:790 [139734346802944] logmon: [Profile12] MATCH [Adapter Connection] on line 0
Jul 29 15:56:59:790 [139734294353664] logmon: (scan) ORA-00942 offset 0
Jul 29 15:56:59:790 [139734388762368] logmon: lgm: format start
Jul 29 15:56:59:790 [139734262884096] logmon: [Profile4] In WithI18n section [2020-07-29 05:11:09 CEST [SAP.0110.0003I] SyncALETransport - Commit 0AEDE2D03F805F20E8CD1DAF],[ERCPû],[ISO-8859-1],[-1]
Jul 29 15:56:59:790 [139734294353664] logmon: [Profile7] In WithI18n section [2020-07-29 13:34:56 CEST [ISP.0090.0003C] ADPT_FWK_TRANSCO_JDBC_SERVER_IS:ADPT_FWK_TRANSCO_JDBC_SERVER_IS request is OK],[ERCP]],[ISO-8859-1],[-1]
Jul 29 15:56:59:790 [139734357292800] logmon: [Profile13] In WithI18n section [2020-07-29 13:49:54 CEST [ISP.0090.0003C] FrmkErrorHandlingCnx:FrmkErrorHandlingCnx request is OK],[ERCP]],[ISO-8859-1],[-1]
Jul 29 15:56:59:790 [139734304843520] logmon: (scan) Adapter Connection offset 0
Jul 29 15:56:59:790 [139734409742080] logmon: lgm: format start
Jul 29 15:56:59:790 [139734294353664] logmon: (scan) Adapter Connection offset 0
Jul 29 15:56:59:790 [139734304843520] logmon: [Profile8] In WithI18n section [2020-07-29 12:32:01 CEST [TNS.0000.2073I] Inserting doc 58d1pa00eed220vd00003rr3 into data store.],[ERCPû],[ISO-8859-1],[-1]
Jul 29 15:56:59:790 [139734388762368] logmon: lgm: Read File
Jul 29 15:56:59:790 [139734294353664] logmon: [Profile7] In WithI18n section [2020-07-29 13:34:56 CEST [ISP.0090.0003C] ADPT_FWK_TRANSCO_JDBC_SERVER_IS:ADPT_FWK_TRANSCO_JDBC_SERVER_IS request is OK],[ERCPû],[ISO-8859-1],[-1]
Jul 29 15:56:59:790 [139734388762368] logmon: lgm: read the line: [2020-07-29 15:14:51 CEST [ISP.0090.0003C] TeRefExtCnx:getConnectionZExtData request is OK]
Jul 29 15:56:59:790 [139734357292800] logmon: (scan) ORA-00942 offset 0
Jul 29 15:56:59:790 [139734388762368] logmon: lgm: check format start..[0]
Jul 29 15:56:59:790 [139734388762368] logmon: lgm: read the line: [2020-07-29 15:14:51 CEST [ISP.0090.0003C] frmkTeSequenceCnx.connection:getConnectionSequence request is OK]
Jul 29 15:56:59:790 [139734399252224] logmon: [raffinage] In WithI18n section [2020-07-29 13:19:54 CEST [ISP.0090.0003C] FrmkTeTranscoCnx:FrmkTeTranscoCnx request is OK],[ERCPû],[ISO-8859-1],[-1]
Jul 29 15:56:59:790 [139734399252224] logmon: (scan) SAP Listner offset 0
Jul 29 15:56:59:790 [139734409742080] logmon: lgm: FORMAT END START
Jul 29 15:56:59:790 [139734409742080] logmon: (scan) SAP Listner offset 0
Jul 29 15:56:59:790 [139734409742080] logmon: [italy] In WithI18n section [2020-07-29 11:48:16 CEST [ISU.0000.9999E] Unable to initialize resource bundle manager (TeFL01994FromRetail_IT.adapters:FL01994Trigger).Can't find bundle for base name TeFL01994FromRetail_IT.adapters:FL01994Trigger, locale en_US],[ERCP|],[ISO-8859-1],[-1]
Jul 29 15:56:59:790 [139734409742080] logmon: (scan) Adapter Connection offset 0
Jul 29 15:57:00:267 [140419014129408] Controller: Max. restarts reached for probe 'logmon' (command = logmon)

------------------------------
HCL Technologies
------------------------------

[Gene Howard]

This would appear to be a unix installation based on the file paths.
what version of UNIX/linux is this installed on?
Check for CORE files on the system and see if logmon is crashing.

------------------------------
Gene Howard
Principal Support Engineer
Broadcom
------------------------------

[Sid]

HI Gene,
They all are unix servers

------------------------------
HCL Technologies
------------------------------

[Garin Walsh]

It helps a lot to have the OS that you are using because there are some dependencies there.

Logmon, since the 3.8x or so versions, has been pretty twitchy. I also see it frequently failed.

There are several causes in my experience - if you use date primitives, they are OS specific - most of the Linux primitives won;t work on Windows and vice versa - behavior is undefined if you accidentally mix them but usually results in a crash.

If you are using wild cards like * check to make sure that it isn't matching a huge number of files - it's perfectly OK to use *.log as a pattern so long as you don't create a log file hourly. After a couple months, logmon will fault with not being able to process those files.

Make sure that the files you are monitoring contain no special characters. Older version of logmon seemed able to read pretty much anything. Today if it isn't plain text you are in trouble.

In the logmon probe directory there's a file "logmon.dta" where logmon keeps track of its current position in the files it's reading. I've seen this file get corrupted - especially if the file is deleted and recreated often. If you don't care about reprocessing files, you can stop logmon, delete this file, then restart.

[David Michel]

[italy] In WithI18n section [2020-07-29 11:48:16 CEST [ISU.0000.9999E] Unable to initialize resource bundle manager (TeFL01994FromRetail_IT.adapters:FL01994Trigger).Can't find bundle for base name TeFL01994FromRetail_IT.adapters:FL01994Trigger, locale en_US],[ERCP|],[ISO-8859-1],[-1]

could be a problem with localization or maybe the java_jre deployed there. 

------------------------------
Support Engineer
Broadcom
------------------------------

[Garin Walsh]

"In WithI18n section " indicates that the failure is happening (possibly) in trying to translate the text to the specified locality. You might want to try the profile without internationalization.

And as an aside, you may have seen the identifier i18n and wondered what that was. In the ancient days of the computer era, engineers at Digital (responsible for the PDP and VAX computers, the Alpha processor, etc.) got tired of writing long words out and so came up with the idea of abbreviating by writing the first letter followed by the number of letters dropped followed by the last. So you could write "Broadcom" as B6m.

There's actually a bunch of research about how you actually read words that argues that you look at the beginning and end of the word and guess at the small set of possibilities and then refine that list based on the remaining letters. It's one of the things that lets you tolerate incorrect spelling without even noticing.    

[Sid]

"In WithI18n section " indicates that the failure is happening (possibly) in trying to translate the text to the specified locality. You might want to try the profile without internationalization - How do i do that ... for a particular profile ?

------------------------------
HCL Technologies
------------------------------

[Sid]

file attached

------------------------------
HCL Technologies
------------------------------

Attachment(s)

EAI-logmon.txt   49 KB 1 version

[Sid]

Logon with debug 5

------------------------------
HCL Technologies
------------------------------

Attachment(s)

EAI-logmon-Max.txt   280 KB 1 version

[Sid]

there is no java jre on deployed the server

------------------------------
HCL Technologies
------------------------------

[David Michel]

as a test
deactivate logmon
rename logmon.cfg
deploy logmon again
activate 
If it remains running you know there is a problem with the cfg

However this does not correspond to the original problem description:
When i deactivate/active probe and validate security it comes  up. 
validate security effects controller.cfg not logmon.cfg.

------------------------------
Support Engineer
Broadcom
------------------------------