Tech Tip: How to tell whether a TCP port is blocked or not listening
Technical Tip Description:
If your OneClick server can't connect to your Main Location server, then when you plug the URL to the OneClick server into your browser, your browser will give you an authorization failure. If you attempt to connect to a OneClick server, and the tomcat port is blocked, your browser will say that the connection is refused. There are other more subtle manifestations. If port 14001 on the OneClick server is blocked for example, you will be able to open a OneClick client, but your alarms will not update. These are just a few examples.
Ports can be blocked by firewalls, or by IPTables on a Linux machine or Windows firewall, or host intrusion prevention software, for example. It could also be that the packets are getting dropped on the network, or the server is simply not listening on the port on which it is expected to be listening. Log files and other troubleshooting steps could lead you to determine that a socket cannot be established.
A packet capture will definitively tell you whether the SYN packets are getting to the server, and whether the ACK packets are getting to the client. If you can't get a packet capture, try netcap, or nc, on a Linux machine. If you have a Linux client machine, you can use nc to determine whether the server is listening on the desired port. Consider a SpectroSERVER that is trying to connect to a OneClick server with the address 192.168.5.1. In this context, the SpectroSERVER is the client, and the OneClick server is the server. From the SpectroSERVER, run:
nc -z -v 192.168.5.1 14001
The z says to test the port. The v is for verbose output. Without the v you will get no output. Here is what you could see:
connect to 192.168.5.1 port 14001 (tcp) failed: Connection refused
To get even more data, you can run nc on the server side at the same time. First you will need to stop the process that is ostensibly listening on the respective port (netstat -an would tell you). Then use nc to listen on the port:
nc -v -l 14001
and go to the SpectroSERVER and run:
A successful connection would result in this output on the SpectroSERVER:
Connection to 192.168.5.1 14001 port [tcp/*] succeeded!
and this on the OneClick server:
port 14001 [tcp/*] accepted
If you don't see any output on the OneClick machine, you can infer that the packet didn't get there. It was blocked somewhere along the way.
Netcat can be added to a Solaris machine as a package.
There are no equivalent out of the box Windows commands, but there are freeware utilities available, such as "Why Can't I Connect" (not endorsed by CA) that serve the same purpose.
A socket in SYN_SENT could eventually enter an ESTABLISHED state, or it could just go away, of the server doesn't send an ACK; it just needs to wait for the timeout. In Windows, the default timeout is 3 seconds. 2 seconds for RedHat. It's difficult to infer from a timeout exactly why the socket didn't get established though - it could be network latency, or a firewall, etc.
If anyone is interested on TCP sockets and performance, I had written this a while ago:
Tech Tip: TCP Sockets and Spectrum Performance
Also, Wikipedia has a great page for anyone new to TCP concepts, or people who want to extend their knowledge of it. It has great diagrams showing the handshake process and definitions of all of the states. Transmission Control Protocol - Wikipedia, the free encyclopedia
There are some OOB options when running Spectrum on a Windows machine.
For example, you can use "netstat -ano <refresh time> | findstr SYN_SENT"
The refresh time determines how often the monitoring is refreshed. For example, if you were to use "3" then every 3 seconds it would check for sockets in a SYN_SENT state.
If you see that a socket is in the SYN_SENT state, that means that the traffic is being dropped, ignored or blocked. A working socket should have an ESTABLISHED state.