Endpoint Protection

 View Only
Expand all | Collapse all

Unexpected server error after upgrade to 14.3.1148.0100

  • 1.  Unexpected server error after upgrade to 14.3.1148.0100

    Posted Aug 06, 2020 06:02 PM
    Edited by Aleksandar Devecerski Aug 06, 2020 06:08 PM
      |   view attached

    Hello all


    Just noticed constant "Unexpected server error" messages in SEPM, Admin, Servers frame.


    Upgraded SEPM to v14.3 MP1 few days ago, with couple of selected clients (one on server, the rest on selected of Win10/Win7, 32/64 bit machines). No problems at all, everything went smoothly, as always. After day or two of standard use for these, without any issues, deployed MP1 install packages to the rest of the client groups.

    Everything seems to be working fine, I don't see anything else besides this error message.

    Symantec Diagnostic Tool screen reports one failed communication test. 3 warnings I think are not relevant (one for IE, the other one for not having free 100 Gbs on system partition - there is 50 and final one for DHCP enabled on NIC - system has 2 network interfaces, one in use with static IP address, the other disabled, set to DHCP auto). Everything else is green.


    "Click here for solution" gets me to the online SEP Installation and Administration Guide, couldn't find anything there.

    What is this error in Diagnostic Tool and could it be responsible for "Unexpected server error"?
    scm-server-0.log, attached, contains some "SEVERE" and "WARNING" items, but I'm not really sure what to do about them.

    SEPM seems to be downloading definitions correctly


    Anyone else seeing this or it's just my system?
    Any suggestions?

    Regards

    Attachment(s)

    log
    scm-server-0.log   27 KB 1 version


  • 2.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Broadcom Employee
    Posted Aug 06, 2020 06:16 PM
    Did you happen to enable the new Audit feature in External Logging after upgrading SEPM to 14.3 MP1?





  • 3.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Aug 07, 2020 02:10 AM
    Edited by Aleksandar Devecerski Aug 07, 2020 03:16 AM
    Didn't make any changes to SEPM after update. This way, if anything happens, I know that new version is the culprit.

    Edit 1 hour later: "Enable Transmissions of Logs to a Syslog Server" -> OFF and Unexpected server error messages stopped.

    I haven't messed with external logging after upgrade, this has been on for at least 6 months (not sure exactly when I turned it on, but it was definitely before Covid-19 lockdown started, which was in March here in Serbia), but I definitely haven't seen "Unexpected server error" message in the console.
    Investigating through syslogs, trying to find out when/how errors started, I've hit syslog limits (configured like this SEPM generates a lot of data), but it seems they started pretty much immediately upon turning on external logging.

    Thanks for the pointer where to look, John
     



  • 4.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Broadcom Employee
    Posted Aug 07, 2020 10:12 AM
    I would get a case opened for investigation. 






  • 5.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Sep 14, 2020 05:19 AM
    Hi. Same problem here. I disabled the "Audit Log" and the problem resolved, our SIEM resumed the log reception.


  • 6.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Nov 25, 2020 10:39 AM
    We are having this same issue.  Adrian, did you say your SIEM still got the logs even after turning off "Enable Transmission of Logs to a Syslog Server"?


  • 7.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Dec 09, 2020 09:47 AM
    We are having the same issue.  AFter upgrading to 14.3.1148.0100


  • 8.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Broadcom Employee
    Posted Dec 09, 2020 02:19 PM

    Should be resolved in 14.3 RU1. 

    After upgrading to Endpoint Protection Manager 14.3 MP1 and enabling Audit Log in External Logging Unexpected Server Errors occur every 30 seconds (broadcom.com)



    ------------------------------
    John Owens
    Strategic Support Engineer | Symantec Enterprise Division (SED)
    Symantec
    United States
    ------------------------------



  • 9.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Dec 10, 2020 02:40 AM
    Edited by Aleksandar Devecerski Dec 10, 2020 02:40 AM
    Nope. Still happening here.



  • 10.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Broadcom Employee
    Posted Dec 10, 2020 10:26 AM
    That may be a different issue. Does turning of Audit Logging fix it?  You can see the full error if you go to Monitor > Logs > System > Server Activity logs. Please post the full error when you get it.

    ------------------------------
    John Owens
    Strategic Support Engineer | Symantec Enterprise Division (SED)
    Symantec
    United States
    ------------------------------



  • 11.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Dec 10, 2020 10:32 AM
    Unchecking Audit logs helped us.  We haven't gone to RU1, yet, so I can't say for that.  However, audit logs are a necessary control for us so I sure hope RU1 can fix not being able to send audit logs to our SIEM.


  • 12.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Broadcom Employee
    Posted Dec 10, 2020 10:48 AM
    Yeah. For that issue, 14.3 RU1 has the fix. So you will be able to send audit logs to your SIEM once you upgrade.

    ------------------------------
    John Owens
    Strategic Support Engineer | Symantec Enterprise Division (SED)
    Symantec
    United States
    ------------------------------



  • 13.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Dec 10, 2020 02:13 PM
    Edited by Aleksandar Devecerski Dec 10, 2020 02:15 PM
      |   view attached

    Hello John

    Error messages start when I turn external logging ON. And stop when logging is OFF.
    The same as with 14.3 MP1.



    External logging was on some 5 minutes this morning 8:25 - 8:30 (not sure exactly) and 5 minutes just now, from 19:43 - 19:48.
    Attached you can find todays mentioned event log ('error and above' severity).

    There are other errors, 90 in total.
    While external logging is ON (2 periods of approx 5 minutes each) errors are these

    Error detail while external logging is ON


    and 30 seconds apart (sorry, but exported report does not contain Stack Trace field).

    These are error details while external logging is OFF. Errors are 15 minutes apart

    Error detail while external logging is OFF



  • 14.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Broadcom Employee
    Posted Dec 10, 2020 02:17 PM

    Ok. I see it:

    To confirm you are running SEPM 14.3 RU1 and have Audit log disabled and this error continues?

    12/10/2020 19:47 Error An unexpected exception has occurred Unexpected server error. Unexpected server error. Unexpected server error. "java.lang.StringIndexOutOfBoundsException: begin 0, end -66, length 124222 at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3319) at java.base/java.lang.String.substring(String.java:1874) at com.sygate.scm.server.task.ExternalLoggingWorker.splitLongMessage(ExternalLoggingWorker.java:3312) at com.sygate.scm.server.task.ExternalLoggingWorker.getPolicyEventLogData(ExternalLoggingWorker.java:2379) at com.sygate.scm.server.task.ExternalLoggingWorker.handleLog(ExternalLoggingWorker.java:600) at com.sygate.scm.server.task.ExternalLoggingWorker.run(ExternalLoggingWorker.java:429) at java.base/java.util.TimerThread.mainLoop(Timer.java:556) at java.base/java.util.TimerThread.run(Timer.java:506) com.sygate.scm.server.util.ServerException: Unexpected server error. at com.sygate.scm.server.util.ServerLogger.log(ServerLogger.java:506) at com.sygate.scm.server.util.ServerLogger.log(ServerLogger.java:470) at com.sygate.scm.server.util.ServerLogger.log(ServerLogger.java:466) at com.sygate.scm.server.util.ServerLogger.log(ServerLogger.java:462) at com.sygate.scm.server.task.ExternalLoggingWorker.run(ExternalLoggingWorker.java:463) at java.base/java.util.TimerThread.mainLoop(Timer.java:556) at java.base/java.util.TimerThread.run(Timer.java:506) Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end -66, length 124222 at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3319) at java.base/java.lang.String.substring(String.java:1874) at com.sygate.scm.server.task.ExternalLoggingWorker.splitLongMessage(ExternalLoggingWorker.java:3312) at com.sygate.scm.server.task.ExternalLoggingWorker.getPolicyEventLogData(ExternalLoggingWorker.java:2379) at com.sygate.scm.server.task.ExternalLoggingWorker.handleLog(ExternalLoggingWorker.java:600) at com.sygate.scm.server.task.ExternalLoggingWorker.run(ExternalLoggingWorker.java:429) ... 2 more " s-006 WBM



    ------------------------------
    John Owens
    Strategic Support Engineer | Symantec Enterprise Division (SED)
    Symantec
    United States
    ------------------------------



  • 15.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Dec 10, 2020 02:22 PM
    Edited by Aleksandar Devecerski Dec 10, 2020 02:24 PM

    No. That error happens only while external logging is ON and repeats every 30 seconds

    Yes. SEPM version is 14.3 RU1



  • 16.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Broadcom Employee
    Posted Dec 10, 2020 02:24 PM
    Ok. So if you turn off Audit in external logging settings do the errors stop?

    ------------------------------
    John Owens
    Strategic Support Engineer | Symantec Enterprise Division (SED)
    Symantec
    United States
    ------------------------------



  • 17.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Dec 10, 2020 02:26 PM
    Yes


  • 18.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Broadcom Employee
    Posted Dec 10, 2020 02:27 PM
    I would open a case for this.  14.3 RU1 SEPM should have resolved the issue.

    ------------------------------
    John Owens
    Strategic Support Engineer | Symantec Enterprise Division (SED)
    Symantec
    United States
    ------------------------------



  • 19.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Mar 22, 2021 01:22 PM
    Hey guys I have the same problem in SEPM version 14.3.3385.1000 Did you manage to solve this?


  • 20.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Apr 30, 2021 11:45 AM
    Having the same issue in 14.3 RU1 build 3384.


  • 21.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Apr 30, 2021 12:46 PM
    Hey guys
    
    I'm having the same problem.
    
    Does anyone know if with the new version 14.3.3580.1000 this is fixed?



  • 22.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Broadcom Employee
    Posted Apr 30, 2021 12:55 PM

    There is an issue being fixed in 14.3 RU2. Eta May/June.

    Does turning off Audit Log fix your issues?



    ------------------------------
    John Owens
    Strategic Support Engineer | Symantec Enterprise Division (SED)
    Symantec
    United States
    ------------------------------



  • 23.  RE: Unexpected server error after upgrade to 14.3.1148.0100

    Posted Apr 30, 2021 01:01 PM
    >>...Does turning off Audit Log fix your issues?

    It does for us (14.3 RU1 build 3384).