I have 10.5.2.24 with SP2 installed. I am not able to connect to MOM using workstation. Even Webview is also not able to connect to MOM.
MOM logs I am getting following:
5/22/18 10:47:53.073 AM EDT [DEBUG] [Acceptor Helper 1] [Manager.AcceptorHelper] Error handling incoming Connection. This may happen normally when a Workstation logs in: java.net.SocketTimeoutException
apmadm@emkhi23971:/opt/CA/APM/10.5.2.24/bin> ./WVCtrl.sh startUsing APMHOME: /opt/CA/APM/10.5.2.24Using JAVA_HOME: ../jreUsing EM Host: emkhi23971.acc.american.comUsing EM Port: 5001MOM/EM is running..JVM PID is 20182./WVCtrl.sh start: Starting APM WebView...(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)328Problem while starting WebView server. Please check /opt/CA/APM/10.5.2.24/logs/IntroscopeWebViewConsole.log for details../WVCtrl.sh start: APM WebView could not be started
Try clearing the work cache on the EM by deleting the contents of <EM_HOME>/work.
This is likely the problem if you were previously running as root and then moved to a non-root user.
Also, clear your browser cache.
I am running both MOM and Webview as apmadm since I install. I clear the cache and restart both MOM and Webview /ATC but still having same problem. My Workstation is installed in my laptop so don't need clear the browser cache.
any other suggestion because I am still getting same
Stop the MOM, clear the work cache, change root logger to DEBUG, restart MOM and attempt to connect. Post results of the login attempt here.
In addition to what Hiko posted, the KB below will show you how to clear all the cache so that Workstation and Webview can connect.
How do I clear the cache on a APM Enterprise Manag - CA Knowledge
Here you go the logs
1. From your laptop, telnet to <IP_EM>:5001
2. The services were iniciated with user root?
Thanks; I'll report back soon.
And the logs Webview?
Not likely related, but you should check your Management Module that has this action in it and remove it temporarily:
Richard Here you go the WebView logs
5/21/18 12:27:28.080 PM EDT [INFO] [WebView.HealthMonitor] EM/MOM status is unavailable5/21/18 12:27:28.081 PM EDT [INFO] [WebView.Login] Logged out user "WilyWebView"5/21/18 12:27:28.084 PM EDT [ERROR] [WebServer.RemoteClientConnectionManager] Unable to connect to EM emkhiabc.un.america.com:5001,com.wily.isengard.postofficehub.link.net.HttpTunnelingSocketFactory - java.net.ConnectException: Connection refused (Connection refused)5/21/18 12:27:28.085 PM EDT [ERROR] [WebView.Login] Error while releasing session token.5/21/18 05:02:49.213 PM EDT [INFO] [WebView] The WebView application has successfully stopped.5/21/18 05:02:49.230 PM EDT [INFO] [org.mortbay.log] Stopped SocketConnector@0.0.0.0:80805/21/18 05:14:41.657 PM EDT [INFO] [WebServer] Starting Web Application Server5/22/18 10:53:16.097 AM EDT [INFO] [WebServer] Starting Web Application Server5/22/18 11:55:44.241 AM EDT [INFO] [WebServer] Starting Web Application Server
Shutdown the EM, remove the module, and restart.
After the EM has fully started, add the module back using the 'deploy' folder and fix the alert action.
This particular field solution requires a EM and workstation plugin. Make sure you have both. The EM plugin goes to <EM_HOME>/product/enterprisemanager/plugins. The WS plugin goes to <EM_HOME>/ws-plugins.
Right now I move this MM to different folder and restart MOM. still facing same problem
Have you tried what Richard suggested and tried to telnet from your WV server to EM on port 5001?
What does 'netstat' say? Is the port stuck open? If so, do you have the commandline tool 'fuser' installed so you can clear the TCP port?
5001 port is open
Trying 10.651.64.57...Connected to emabc31s016.Escape character is '^]'.
Are your webview and EM on the same server or different servers?
Check your webview properties file for the tcp.host, tcp.port and webserver.tcp.port settings and see if they are set properly. If using localhost, try using the hostname or IP.
They both are running on different machines. It was working for last 3 -4 month but suddenly start having problem. I think culprit is MOM because it is not allowing any workstation connection.
Have you recycled MOM and Webview? If so and you still see the same issue, then I recommend opening an issue to debug it further.
Yes the root cause may be the MOM performance. Recycling MOM per Matt's advice should hopefully resolve it at least temporarily. However reviewing the perflog.txt (How to interpret the values of Perflog.txt fields. - CA Knowledge) or creating a support case for deeper review would be a good idea to confirm root cause and prevent future occurrences.
I run "netstat -an | grep 5001" and getting more then 50 results. some of them are showing FIN_WAIT2. My other MOM who are running fine are showing only 6 or 7 results. yesterday I change the port from 5001 to 5002 and I was able to connect to MOM using workstation also Webview was connected fine.
I Google it and found that the number of ports are exhausted Normally this type of problem occurs because the web application is opening connections under the covers and then put the port into TIME_WAIT and thereby.
My question is is there a way I can forcefully close the connections. I asked our Linux admin to reboot the machine but it didn't help.
If you are able to install the package ‘psmisc', it comes with a commandline tool called ‘fuser' that can be used to free those ports without rebooting your server.
I know on you're on SuSE, so I'm not sure how to do that on that platform. For CentOS/RHEL, you need to install the following: ‘epel-release psmisc'.
Once installed, just run the following command:
fuser -k -n tcp 5001
Hiko_Davis its help and close all the connection also kill MOM process but when I start MOM all the tcp 5001 connection are again showing the same. Is there any other way to fix it?
Can you account for each connection? Should be one for each Collector, WebView, and workstation.
Looks like we'll need to take a deeper look at your EM and OS configurations.
Please open a ticket and make sure to reference me as your contact.
Please upload your properties files and logs.
shaja15 : FYI
Support case # 01095393 - Connecting to Enterprise Manager
There will also be many open sockets for the EM->APM DB connection.
Testing with my 10.7 MOM immediately after startup shows 24 open sockets from its java process to postgres socket 5432. You can check with command "netstat -nap|grep 5432|grep -i jav|wc -l"
Try installing fuser this way: sudo zypper install psmisc
Hopefully, it will also download any dependencies for you.
This article may be relevant to you when you go to install 10.7 and Infrastructure Agent: https://comm.support.ca.com/kb/systemedge-agent-install-fails-on-linux-with-errors-about-missing-libraries/kb000033669
Check the version of your workstation install and the version of the EM - MOM. If you look in the MOM log (IntroscopeEnterpriseManager.log) there is a start up banner that has the version number. On the Workstation, on the log in screen, it will have the version and build number. Pretty good chance that a version 10.0.x workstation will not connect to a 10.5.2 EM. Check the downloads for 10.5.2.24 sp2 for a new workstation install.
I've been having the same issues for months (support case 01086810) and I'm to the point of thinking that it is the WVCtrl.sh script itself. So the next time this type of issue occurs, I'm going to try to start WebView from binary.
nohup ./Introscope_WebView.bin >> "nohup.out" 2>&1 &
Yea, using this makes it messy to stop WebView since you will need to get the pid (ps -elf|grep webview) and then kill -9 <webview pid>
Also try to run workstation from the MOM http://<mom host>:8081 in IE with Java plugin and click on launch workstation to insure the versions all match up.
Hope this helps,
Yes in general the workstation and EM/MOM versions being used must match to ensure full compatibility because that is the only combination that has been tested. A mismatch of versions may work but not guaranteed because of changes across versions.
The problem start when we are moving agents from one Data Center to another Data Center APM Environment.
I worked with Norris Graves and Hiko_Davis and find out that some DNS server are acquiring these 5001 ports. temp we are moving problematic agent to old Data center APM environment after that I will work with App owners and Linux team to fixing the problem.
Thanks everyone for the help