Clients not updating and Management server showing offline

[Migration User]

I've gone through a number of forum threads and tech support articles on SEP manager/client communications and haven't found a fix for my issue. I have one SEP management server which is running on Windows 2008 Server Enterprise x64. My manager is running on version 11.6. I should note that when this issue started on Tuesday the manager was on version 11.5. After some basic troubleshooting I thought maybe if I upgraded something would get fixed. That is not the case and we were on version 11.5 for a while so it's not an upgrade issue. All clients, whether they are running on a workstation (XP or Win7) or a server (2003 or 2008) are not communicating with the manager server. When I open a client and go to Help and Support --> Troubleshooting the Server shows Offline and there is not green dot over the Symantec shield in the Task Bar. On the management server most of the clients do not show as Online. Some do but when you check the local client it shows Offline. So it appears that all Clients are Offline even though some show to be Online from the Management Server Console. The Last Check-In times are updating but the clients still show offline. So it appears there is some form of communication but I can't execute any commands like updating content from the managemet server or get any clients to show Online.

I checked ODBC settings and I can connect to the DB. I ran the SEP Support Tool and the Management server and Everything passed so I know I do not need to check IIS settings or anything like that, even though I already when through all those docs.My client versions range from 11.4 to 11.6 and none are communicating with the Management Server. We didn't make any major changes recently other than running windows updates on Saturday. I didn't notice the issue until Tuesday. Not sure if thay has anything to do with it? Is anyone aware of a recent MS update that is causing issues with client/manager communication?

Like I said before I've gone through a number of threads for similar issues and went through all the documentation posted in those threads. No dice. Maybe a good place to start is looking at some logs from the Management Server? Any takers? Thanks for you anticipated help.
-Steve

[Migration User]

I Have exactly the same problem, but have not yet found a solution.
Carlos

[Migration User]

I am experiencing the same issue. I have also discovered that when you try to  upgrade the client to 11.0.6, nothing works. I have tried upgrading the client through the migration wizard through the using the upgrade in the installation tab. I will have to roll back to version 11.0.5.330 because this is unacceptable and my customer does not have a fully functioning product.

I don't know if 11.0.6a has fixed this issue.
-Robert

[Thomas K]

Try testing connectivity from a client to the Symantec Endpoint Protection Manager (SEPM), type the following URL In a web browser:


http://<SEPM_Server_IP_or_Machine_Name:Port>/
secars?hello,secars

Also see -Troubleshooting Client Communication

https://www-secure.symantec.com/connect/articles/troubleshooting-client-commuincation

[Migration User]

I read from you're thread that you've done a bit of searching before you started, so obviously this issue must not be straight forward.

So before suggesting anything, could you post the error you are seeing in the Sylink.log files?

The symptom you described, green light on console, but not on the client, can happen when there is a Certificate Mismatch in the Sylink.xml file. But I'm sure by now you've tried the "replace the sylink file" solution.

Here are a few steps that may narrow down the problem area.
1) Find a client, preferably one that shows a green light on the SEPM console, but not on the client.
2) Delete the client from the SEPM console.
3) On the client, click Update Policy
4) Observer whether the client registers with the SEPM server again.

If the client is able to register with SEPM, but not establish healthy communication, it suggest (does not absolutely prove) there is a certificate issue.

Next, let's make validate for sure it's not a certificate issue.
1) In the SEPM console, select a group that contains a client your troubleshooting.
2) Click on Policies tab -> General Settings -> Security Settings
3) Turn off the last option, Enable secure communication between the management server and clients using digital certificates for authentication.
4) Click OK.
5) Wait a moment to make sure the Server has time to generate the policy (It should be very fast).
6) Click the task option, "Export Communiction Settings"

This exports a Sylink file. Use this Sylink file to replace the one on the client. I'm sure you've tried the SylinkDrop solution before. The difference is this Sylink file will have 'VerifySignatures' flag set to ="0". You can verify this by opening the Sylink file.
If this works, then we know there is a certificate issue. I would assume you have a DB vs. on-disk certificate mismatch, which is very rare. But it should be repairable without dropping a Sylink file onto all the clients.

So once again, be sure to post the errors from your Sylink.log. I would suggest searching for the lines that contain "http://" (or https if you're using SSL). The error codes should be below the connection request.

[Migration User]

SEPM uses IIS for client communications, so that is where you should be concentrating your investigation efforts.  If the clients were communicating prior to the Windows Update on Saturday, then find out if anything else was done that day, or since that day, which may have affected IIS, like IIS lock down, or some other GPO that might have been applied to either the SEPM or the clients.   Overall, when things that are working suddenly stop working on a wide scale, that is typically an indication of an environmental change.

[Migration User]

We found the solution.
There was a problem with the redirect function blocked by the firewall. The firewall let pass icmp, tracert but not redirect 5.
We added it to the exception of the firewall, we did a ipconfig /renew and then we refresh the antivirus policies at the workstation and it works.
Carlos

[Migration User]

I actually ended up fixing this issue yesterday afternoon. What exactly resolved this I'm not sure because I tried so many things it's hard to determine what exactly fixed my issue. I was actually in the process of building a new VM because I was going to install SEPM on a fresh server and migrate all the clients over. In the process of building that server something I changed on the exisiting server must have took because I saw clients starting to show online. The last thing I changed was in General Settings of my Policies. In the Security Settings tab the last option is to Enable Secure Communications between clients and management server usign digital certificates. I unchecked that on all policies. That was probably 30 minute to an hour because I saw clients coming online. I don't ever remember checking this option. Perhaps it's checked by default? If that is the case I don't know why all of a sudden it would cause an issue. We have been up and running on SEP 11.x for 8 months or so. Rigth around the time things started working again I reconfigured the ODBC connection for the embedded database. Possibly that helped? I had been messsing with this for a few days and the two things I mentioned in this reply were the only things I changed yesterday so I am guessing one of those two fixed my issue. Unless my management server installation has some issues and pehaps I will have a problem again in the near future.

for those of you still having this issue perhaps just building a new server would be a quicker fix than trying 15 different things and not knowing what fixed what or caused more problems. The process for building a new server and migrating everything over actually seems like a simple and straight forward process.

http://service1.symantec.com/support/ent-security.nsf/docid/2008031204405448

[Migration User]

I would guess that if the 'secure communication' option was the issue, the clients wouldn't be able to talk unless you manually replaced the Sylink file on all the clients. But just in case there is an issue here, it's worth checking... by the way, the "Enable Secure Communications" is enabled by default.
To make sure you don't presently have a certificate issue (which is related to 'secure communication) move a signle client into a new group and try enabling 'secure communaction' in that new group.
If the client disconnects, it would be best to get the issue resolved to prevent any problems down the road.
To resolve the issue, re-import your server certificate under Admin -> Servers -> (click on your local server) -> Tasks: Manage Server Certificate

If the client stays connected, then I would say the 'resolution' was something else.