The workstation needs to be of the same version as the MoM. Means, you will have to get the correct HF 60 version of the workstation to have a seamless integration with the updated MoM. Please ask support to provide you the Workstation for your current installation.
Regards
Original Message:
Sent: Feb 25, 2022 12:59 AM
From: Shiv Choudhary
Subject: Enterprise Manager restarted by administrator error in CEM , EM crashing
Hi All,
We have applied hotfix HF 60 to our MoM server & not applied currently to collectors in Non Prod Env.
getting below error in WebView 1. Error retrieving permissions. Status code: 500
Enterprise Manager and WebView start up with no errors. But When logging into Enterprise Manager Team Center, seeing the following error in the IntroscopeWebview.log and there isn't any data showing up at all under metric view, agent view, map
Also in New APM status console , collectors are not reflecting , I suspect it was due to not applying HF60 patch on collectors or due to some other issue . Also one of our Agent collector is on 10.5 & other TIM collector is on 10.7 version so how do we proceed ?
------------------------------
Shiv Choudhary
India
Original Message:
Sent: Feb 16, 2022 03:48 AM
From: Jörg Mertin
Subject: Enterprise Manager restarted by administrator error in CEM , EM crashing
Hi SHiv,
the Hotfix is to be applied on the EM (Collectors) and MoM only.
From your explanation, you have one MoM, one EM (for Agents) and one EM for the TIM collection service. You have to apply the Hotfix to all these 3, and also on the machine (if separate) running the PostgreSQL Server.
You also have 3 TIM's running. These do not need to have the Hotfix applied. If there are any Hotfixes for the TIMs, it will come in a separate hotfix package.
Original Message:
Sent: Feb 12, 2022 10:30 PM
From: Shiv Choudhary
Subject: Enterprise Manager restarted by administrator error in CEM , EM crashing
Hi Jorg ,
I got the hotfix HF 60 from support . So my question is that we have to implement it in
1) MoM server only
2) or also on Agent collector and TIM collector server
3) or we have to apply this hotfix on TIM server as well ?
as we have one MoM server , one Agent collector server & one TIM collector server & 3 TIM servers which are reporting to TIM collector server . All servers are Physical server in our environment .
I also found in the readme.txt , Configuration steps for the Cluster. So, does this step need to be done?
If yes, when we need to do this before or after installing hotfix. PFA screenshot below. Also let me know is it to be done after applying the hotfix on all 3 servers i.e., including a MoM & an agent collector and TIM collector server .
------------------------------
Shiv Choudhary
India
Original Message:
Sent: Jan 28, 2022 04:24 AM
From: Jörg Mertin
Subject: Enterprise Manager restarted by administrator error in CEM , EM crashing
Hi Shin,
From the case 32979151 logs I see you have not applied Hotfix SP3 yet. That would help because the condition you run into seems to be due to an issue causing the OOM.
If after you applied SP3 this problem still occurs, please see https://knowledge.broadcom.com/external/article?articleId=140703 - just in case you do not apply the memory changes at the right location.
Original Message:
Sent: Jan 28, 2022 04:05 AM
From: Shiv Choudhary
Subject: Enterprise Manager restarted by administrator error in CEM , EM crashing
Hi Jorg,
No error is showing as of now in CEM, only error getting now is Enterprise manager restarted and with every instances of error in CEM getting log message java. lang.OutofMemoryError : Java heap space.
1) I doubt if initial Java heap memory & maximum java heap memory setting needs to be done .
I see that we can change the value of wrapper.java.initmemory & wrapper.java.maxmemory in EMService.conf file. Current value set as below:
# Initial Java Heap Size (in MB)
wrapper.java.initmemory=4096
# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=10240
2) Also i have seen one file is there SmartStorTools.bat.
Edit JAVA_OPTS to increase the heap size if needed. Current value set as below:
set JAVA_HOME=%INSTALLDIR%\jre
set JAVA_OPTS=-Xmx512m
Currently the value set to 512m should we increase or decrease to resolve Java heap space issue
3) Also there is one another file Introscope_Enterprise_Manager.lax where i have seen option to set the lax java option additional values like Xms & Xmx setting.
lax.nl.java.option.additional=-Xms8192m -Xmx8192m -Djava.awt.headless=false -Dmail.mime.charset=UTF-8 -Dorg.owasp.esapi.resources=./config/esapi -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./logs/ -Xss512k
Plz let me know the exact setting needs to be done to all these to resolve java heap space issue if we have 32GB of RAM available in the server .
------------------------------
Shiv Choudhary
India
Original Message:
Sent: Jan 24, 2022 03:37 AM
From: Jörg Mertin
Subject: Enterprise Manager restarted by administrator error in CEM , EM crashing
Hi Shiv,
this could be linked to the other issue you see with the TIM & Defect Storm.
The one thing we do not tend to see, is that the aggregator is spawned as its own instance. Hence requiring the same amount of memory as usually the EM/MoM itself.
This in turn means that if you MoM is using 1Gb, when the aggregator runs (hourly aggregator) it will use one additional GB of Ram.
If you now have the daily aggregator running because it is catching up (due to the TIM not providing data for a while) to handle the statistics and the defects, this 1Gb will also add up. If in the worst case you are at the end of a Month and have also the Weekly and Monthly aggregators running at the same time, you'll have 4 additional aggregators running, each using another 1Gb of RAM.
So - only thing you can do here - if this is a Virtual machine, add enough resources (here RAM and CPU to speed up the process). If not as stated in the other thread (https://community.broadcom.com/enterprisesoftware/communities/community-home/digestviewer/viewthread?MessageKey=32ffb92e-41ea-4af0-ae83-42b19dff0c8a&CommunityKey=be08e336-5d32-4176-96fe-a778ffe72115&tab=digestviewer&bm=32ffb92e-41ea-4af0-ae83-42b19dff0c8a#bm32ffb92e-41ea-4af0-ae83-42b19dff0c8a) make sure you stop the defect storm condition so the MoM/EMs can run normally.
Note: I also see that you have one of the first releases. Using (applying) SP3 at least would help increase stability.
Original Message:
Sent: Jan 22, 2022 02:10 AM
From: Shiv Choudhary
Subject: Enterprise Manager restarted by administrator error in CEM , EM crashing
Our MoM server crashes intermittently , service getting restarted on it's own & comes up.
Didn't know exactly why this is happening . attaching logs & screenshot from the MoM Server & we are using APM 10.7.0.45 version . Case has also been raised 32979151 with Broadcom.
------------------------------
Shiv Choudhary
India
------------------------------