I forgot to mention one actually quite common source of delays:
PAM has a socket filter agent (SFA) component that can be installed on Windows and some UNIX/Linux operating systems. This agent listens on port 8550. When PAM connects to Windows or UNIX/Linux devices, it will always first try to connect to port 8550 to notify the SFA, should one be installed and running, that a new connection is coming in from PAM. This is done unconditionally. If your network filters communication to port 8550, the connection attempt will time out after 10 seconds, and only after that will PAM connect to port 22 (for SSH) or 3389 (for RDP). The solution is to either open communication for port 8550, or reject (rather than drop) connection attempts to that port so that they will fail immediately rather than time out.
You can check yourself on whether this problem is present in your environment by looking at the SPFD log:
Go to the Configuration > Diagnostics > Diagnostic Logs > Download page.
Click on the DOWNLOAD button to the right of the "SPFD Logs" label. Note that this file could be several 100 MB in size. If needed split the file and edit only a part of it.
Log lines have format
<date> <time> <PID> <log level> <message>
Look for string "ERROR probeSFA". Note the value of <PID> in this line.
Now look backwards for <PID>. If the connection timed out, you will see a "Connect timeout" message just prior to it.
Here is one example:
2019-07-24 16:45:40 27801 INFO run: Policy: sessionID=...
...
(<multiple log lines with other <PID> values for the next 10 seconds)
...
2019-07-24 16:45:50 27801 ERROR open: open: Connect timeout. (Operation now in progress)
2019-07-24 16:45:50 27801 ERROR probeSFA: Fail to probe SFA on target(mypamtarget.my.domain:8550)
You can see that the "Connect timeout" messages comes 10 seconds after the previous message logged for this PID. So a user will observe at least a 10 second delay after launching an access method like SSH to this target device.
Original Message:
Sent: 07-22-2019 04:29 PM
From: Ralf Prigl
Subject: CA PAM 3.2.4 is slow while accessing SSH applet and also clients
Hello, There can be many factors involved for a connection through PAM vs a direct connection to the target device:
- Different route. In general the route through PAM will be longer. You can use the Devices > Networking Tools page to check on the routes between PAM client and PAM, and PAM and target device.
- Session recording. If the session is being recorded, there will be overhead upfront to initialize session recording. Ongoing performance during the session will be impacted by the quality of the connection between PAM and the recording share.
- PAM clustering. When PAM is clustered, information on the new session is replicated to other cluster nodes during initialization. This is significant specifically in combination with the previous point, session recording. One known problem is with significant delays when the cluster is on and a DNS server is configured that does not respond promptly to DNS queries. Just one out of multiple configured DNS servers can cause delays.
- Command and/or socket filtering. If filters are defined in the access policy, they can cause some overhead in the communication with the device.
- PAM appliance resource usage. If CPU or memory usage is high on the PAM appliance, or allocated resources are insufficient for virtual instances, it may slow down connections to target devices.
Original Message:
Sent: 07-22-2019 06:28 AM
From: Ramesh Dara
Subject: CA PAM 3.2.4 is slow while accessing SSH applet and also clients
Hi All,
We are facing issues while accessing SSH applet and also clients securecrt is very slow and ssh applet is taking to connect 10-12 seconds.
Our PAM version is 3.2.4, also when securecrt access and when executing commands it running very slow comparing with out PAM when directly login to securecrt after commands executing very fast.
Please suggest whether we have to do any thing changes in PAM or in Network side.
------------------------------
regards
Ramesh
------------------------------