SEP system tray icon warns of "multiple issues" following installation of January 3rd, 2018 Windows Security Update (KB4056892)

[Matt Mc]

Hello Everyone,

As most of you already know, SEP 12.1 and 14 clients which have had Windows Update KB4056892 installed have been experiencing an issue, where an alert on the SEP system tray icon warns of "multiple errors".  This issue is described in the following article:

Endpoint Protection system tray icon reports there are multiple errors after updating ERASER to 117.3.0 and Microsoft Update KB4056892
http://www.symantec.com/docs/TECH248552

SEP 12.1 RU6 MP5 or earlier clients can also encounter the following issue:

ccSvcHst.exe crash after applying the January 3rd, 2018 Windows Security Update to a system running SEP 12.1.x
http://www.symantec.com/docs/TECH248558

Please note that SEP 12.1 RU6 MP6 or later will still experience the system tray icon error behavior, but will not encounter a ccSvcHst.exe crash as a result of applying the Windows KB. 

At this time we do not require additional data for either of the above issues.  While anyone affected by either issue is of course welcome to open a case with our Support team, simply subscribing to the above articles will ensure you receive any updates as we determine additional issue details, and any steps needed to resolve these issues.  If you do choose to contact Support, please be aware that queue volume is extremely high at this time.  Thank you. 

Our Developers have provided the following update to offer some clarity around these issues:

These issues are unrelated to the ERASER 117.3.0 Engine update. These issues impact all Operating Systems that received the Windows Security Updates released on January 3rd, 2018.

Ongoing investigation indicates that the tray icon error issue described in TECH248552 has no functional impact on the SEP client with regards to our protection technology stack, but out of an abundance of caution, we are recommending that SEP 12.1/14.0 customers please hold off on applying the Windows Security Updates released on January 3rd, 2018 until our investigation has concluded. 

[CanMan]

Matt - any update please? After opening my case and it languishing for hours I;ve logged on this morning to only find a cryptic POA that seems to imply I'm not to expect any update until late on the 8th January?

[Matt Mc]

@CanMan - I'm continuing to monitor, and our Devs are continuing non-stop work on this issue.  As updates are available, they will be provided in the article.  I will also attempt to post anything new here. 

As for your case POA, at this point we're all waiting for Dev for new information, and the TSE working on your case is likely just being conservative as far as when to expect the next update.  The Devs are keenly aware of the urgent need for new information, or, even better, a solution.    

[TonyS]

Thank you for the update, hopefully we will have something from Symantec soon before Monday where the new term starts.

Sending best regards to all Symantec Dev team working on this.

[Migration User]

Yes - I'll echo that - whilst the situation that we find ourselves in is extremely frustrating - waiting for answers. I have sympathy for the poor guys at the coalface trying to resolve this.

It would be good to have a resolution today, we have a limited patch window until our systems are Live Monday Morning (UK). Luckily I have a Window of oppertunity this weekend - but then we are looking at a week exposed until the next patch window Next Weekend.

 

 

[Matt Mc]

@Tony & P G - Thanks to you both for your kind comments, they're much appreciated.  I've passed this link along to the Devs so they could see them as well. 

[CanMan]

Thanks for reassuring us its being worked on.

[hytekj]

Thank you for the update!

[Matt Mc]

Thanks to all of you for your patience.  I'm continuing to watch for any news from the Dev team, and will keep you posted.  Please ensure you also subscribe to both of the articles, in the event they are updated first.  Thank you. 

[Jack Benson]

I am having the same issue - not sure if there is another way than to post to get email updates on this thread

[Matt Mc]

@Jack Benson - Please subscribe to the tech article on the issue:

Endpoint Protection system tray icon reports there are multiple errors after updating ERASER to 117.3.0 and Microsoft Update KB4056892
http://www.symantec.com/docs/TECH248552

Posting to a thread here will subscribe you to it, but you can also expand "Subscriptions" at the bottom of the first post in any thread and click "this post".  Thank you.

 

[CanMan]

The continued silence is disturbing. 24 hours until CEOs across the world will be expecting many of us to be applying these patches.....

[S_K]

Hi Matt,

As this article https://support.symantec.com/en_US/article.TECH248558.html says:

Applies to SEP 12.1 clients versions 12.1 RU6 MP5 and earlier

does it mean on SEP 12.1 versions higher than RU6 MP5 there is no issue and it is safe to proceed with Windows patching?

 

 

[S_K]

Hi Matt,

As this article https://support.symantec.com/en_US/article.TECH248558.html says:

Applies to SEP 12.1 clients versions 12.1 RU6 MP5 and earlier

does it mean on SEP 12.1 versions higher than RU6 MP5 there is no issue and it is safe to proceed with Windows patching?

 

 

[Migration User]

I am HOPING this gets resolved BEFORE tomorrow's start of the work day.  You are absolutely correct, CEO's and senior mgmt will not be happy about this.

[andymease]

I'm now hearing there will be a hotfix release for this issue

[Migration User]

I still haven't seen any update to the TECH248552 article - where are we? The weekend update windows has now closed for me.

I am a bit concerened that there hasn't been any official update from Symantec.

[Matt Mc]

@S_K - SEP 12.1 RU6MP6 will still experience the tray icon issue, but will not encounter the ccSvcHst.exe crash. 

Regarding both issues, Dev's investigation has concluded. 

For customers that are running 12.1 RU6 MP5 or earlier, our recommendation is to upgrade to a later release prior to applying the Windows Security Update from January 3^rd, 2018.

For customers that are running 12.1 RU6 MP6 or later, they will encounter the system tray icon issue previously outlined in TECH248552.

While the issue outlined in TECH248552 has no functional impact on the protection technology of the SEP client, our recommendation will be for customers to pick up one of the hotfix releases that addresses that issue prior to applying the Windows Security Update.

Further details on the hotfix builds to correct the tray icon issue, and their availability, will be forthcoming as soon as possible.  We should be seeing updated info in the tech articles shortly as well.  Thanks everyone. 

[S_K]

Thanks for the update Matt

[Michael Tremmel]

I have many clients over 150 that have already applied the KB4056892 I sure hope the hotfix can be applied without having to do a rollback!!!!!

[Michael Tremmel]

I see that there has been an update to the tech article:

At this time, this issue has no functional impact on the protection technology of the SEP client.  However, Symantec plans to release a hotfix to address the issue, and recommends that the Microsoft Windows Security Updates released on January 3rd, 2018 updates not be applied to systems until a hotfix is available for the affected versions.

I wonder what the time frame is for the Hotfix release???

[Michael Tremmel]

@Matt MC

 

do you know if the hotfix will be pushed out via liveupdate or is this going to be a manual process??

[Matt Mc]

Hello,

Both of the following articles have been updated with the latest info and recommendations:

Endpoint Protection system tray icon reports there are multiple errors after updating ERASER to 117.3.0 and Microsoft Update KB4056892
http://www.symantec.com/docs/TECH248552

ccSvcHst.exe crash after applying the January 3rd, 2018 Windows Security Update to a system running SEP 12.1.x
http://www.symantec.com/docs/TECH248558

For issue one, involving the tray icon errors, there will be info on the forthcoming SEP hotfixes added to TECH248552 as it becomes available. 

For issue two, with the ccSvcHst.exe crash, please ensure 12.1 clients are upgraded to at least RU6MP6 to avoid this issue.  This is stated in TECH248558. 

To confirm what was mentioned earlier, the tray icon issue in TECH248552 has no functional impact on client protection technology.  However we do recommend waiting until the SEP hotfix for this issue is available before applying the Windows Security Updates released on January 3rd, 2018. 

For those who have already installed KB4056892 (@Michael Tremmel, and many others I'm sure), I currently have no indication that this would prevent applying the forthcoming hotfixes for SEP, but I am seeking further information from Development on this question. 

Thank you. 

[Torb]

Will the hotfix be released through Liveupdate or require an agent upgrade? Side question: Is CSP/DCS:SA affected by the meltdown/sceptre patch?

[Torb]

How severe is the 12.1 bug? Will the clients always crash or just sometimes? I have alot of customers running 12.1 in all versions that haven’t been upgraded for various reasons. Are they now completly unprotected? Will I see them again in SEPM or will we slowly lose control as Agents are purged from SEPM and they will remain unprotected forever?

[Migration User]

@Matt Mc

If the recommendation will be for customers to pick up the hotfix release prior to MS update, what will be the recommendation for those of us who are also customers and have already applied the MS update?  Will we be able to just apply the hotfix or will we have to first rollback the MS update?

[천 영철]

Hello matt-mc
We have had problems with many PCs since the update.
As you know, there is no problem with the vaccine operation, but the green light does not come on.
Users are confused.

Our company is training to check the "green" fire every day.

How do I resolve it?

[ajhay.siingh]

HI Matt,

Thanks for this Information about this update.

 

Regards

Ajay Singh

 

[Migration User]

Hello,

My company computers have the same issue after the installation of 2018-01 Windows update:

"Product Error requires attention" "There are multiple problems (2)". 

My Symantec product is : Symantec Version 14 (14.0.MP2) build 2415 (14.0.2415.0200)

I have the last definition for all products

My windows PC version: version 1703 (15063) and 1607 (14393) ..

Thank's for your help!

[S_K]

@Symantec support:

1) How will the hotfix be released - via LiveUpdate or as new SEP package?

2) WIll this hotfix also apply for SEP version 12.1 RU6 MP5 and lower or the only fix for them will be upgrade to version newer than 12.1 RU6 MP6? 

 

[ajhay.siingh]

in my view for SEP 12 and 14, both version Symnatec will release new MP version as new bundled AV package.

 

Regards

ajay Singh

[LUIS Cimas]

Hello, Everyone,

I have installed update KB4056896 on windows server 2012. This update is wrong too. You can uninstall this update and the problem is resolved.

See you

[K-Dee]

Thanks for the updates.....

[Scott K]

@ Matt Mc – Thanks for the updates

 

I need answers to the following for my manager and Director of Information Technology:

1) How will the hotfix (TECH248552) be released? Via Live Update? A manual patch?

2) ETA on the release of this hotfix? For example, later today, tomorrow, next week, next month?

3) Since the recommendation from Symantec is hold off on installing the Microsoft Windows Security Updates released on January 3rd, 2018 until after the hotfix from Symantec, what is the recommendation for computers which are affected by this problem? It would be extremely helpful if know before the hotfix is released if we need to uninstall the Microsoft Updates or other manual steps. Also knowing if the hotfix is being designed to work for computers which have the Microsoft Updates already installed would be extremely helpfully to know.

[Michael Tremmel]

@Scott K I received this response earlier yesterday hope it helps a little

For those who have already installed KB4056892 (@Michael Tremmel, and many others I'm sure), I currently have no indication that this would prevent applying the forthcoming hotfixes for SEP, but I am seeking further information from Development on this question. 

[John Owens]

Good Afternoon Everyone,

As soon as we have more information regarding the hotfix we will make it available.  Please continue to monitor the tech documents Matt mentioned.  As soon as we have all of the information we will update this thread as well.

Thanks,
John

[John Owens]

Hi Everyone,

I have some more information regarding the Hotfix. 

• These will be complete builds of the product which will follow all the normal supported upgrade paths and migration steps associated with moving to a new version of the product.
• Client patches will be available for the hotfix releases.
• Liveupdate will NOT be used to distribute the hotfix.
• The 12.1 hotfix will be based off of 12.1 RU6 MP9.

I can assure you we are working as quickly as possible to get the hotfix builds ready and available.  I hope to have a more concrete eta for their availability tomorrow.

Thanks,

John Owens

[Wally]

I see a lot of posts about SEP 12.1.  Does the hotfix scenerio apply to SEP 14 as well?

 

[TonyS]

Morning John,

Thank you for the update. Do we have any update on the new SEP client build? We're very keen to roll it out as soon as possible before we deploy the Microsoft patch.

Thank you to all Symantec Dev for working into resolving this issue.

[deepak sudevan]

It driving me nuts...... have server with windows 2008 and SEPM 14.0.1 and morethan 500 clients. almost 90% affected with same issue.

[S_K]

As far as I heard, HotFix will be also available for version 14

[Migration User]

Any update RE ETA for the hotfix?

[ace1]

Same problem, logged a ticket.

I hope for a very quick release.

Being so systematic, how can a defect like this slip through QA testing?

Doesn't Symantec test the compatibility of a critical (with user and kernel components and many moving parts) software like an antivirus against last OS patches before Microsoft release to the general public?
 

[almost there]

Any ETA available yet? Sorry to ask the same question as everyone else, but we need to be patching these systems. If the issue is simply cosmetic as you say, why is a fix taking so long? 

[TonyS]

See https://www.symantec.com/connect/forums/latest-win10-update-corrupts-sep14?page=1#comment-11953691 for the latest update by Matt Mc @ Symantec

[ℬrίαη]

MS pulled the patch for specific AMD systems:

http://news.softpedia.com/news/microsoft-pulls-kb4056892-kb4056894-updates-due-to-issues-on-amd-systems-519278.shtml

[Matt Mc]

Thanks for mentioning that, Tony.  And please note that what I posted to the other thread (https://www.symantec.com/connect/forums/latest-win10-update-corrupts-sep14) was not a major update, I was just echoing what @John_Owens had posted in this thread last night, for those who hadn't seen it yet.  The following info is still the latest on the hotifxes:

• These will be complete builds of the product which will follow all the normal supported upgrade paths and migration steps associated with moving to a new version of the product.
• Client patches will be available for the hotfix releases.
• Liveupdate will NOT be used to distribute the hotfix.
• The 12.1 hotfix will be based off of 12.1 RU6 MP9.

I'm also contacting the author of http://www.symantec.com/docs/TECH248552 with the request to add these new details to this document.  (Generally speaking the doc would be updated ahead of the forums, and I'd still recommend subscribing to the doc as the most reliable source for new info.) 

As stated in the other thread as well, our sincere thanks to everyone for your patience.  Rest assured that preparing these hotfixes and providing additional details for deployment remains the absolute top priority for the SEP Development team.  We hope to have additional info to share later today. 

 

[darcon]

@ Brian

If read it right, it looks like they are only pulling the patches on AMD affected systems.  

[axemon]

Thanks for the updates Matt and John. It's reassuring to know that you folks are tackling this as fast and as reasonably possible.

 

 

[Baccus]

Matt Mc .. thanks.. i figured it'd come in the form of a new version upgrade.

[Migration User]

None of the normal Patch Tuesday updates will need to be mitigated as this one was, I gather? Or did MS provide the "regular updates" for testing?

[John Owens]

If you are in a position where you must apply the Microsoft patch before the hotfix is available please make sure of the following:

1. Ensure SEP clients are on 12.1 RU6 MP6 or later so you do not run into the ccsvchst.exe crash issue - TECH248558
2. Understand that you may see the SEP icon in the System tray display multiple issues after you patch. These are only cosmetic and will be fixed by the hotfix. - TECH248552
3. You will be able to apply the hotfix after the Microsoft patch is applied to resolve the cosmetic system tray issue.

Thanks,

John Owens

[Migration User]

Hi

I am trying to save myself a phone call with a question regarding versions.  My SEP Manager and all client versions are 12.1.6 (12.1 RU6 MP8) build 7266 (12.1.7266.6800) which I have copied and pasted into this post.

• With my current version can I install the January patch from MS and have no crashes from the SEP client today?
• With my current version is the registry setting that the MS patch is looking for - set in the correct manner?
• With my current version can I install whatever new build of the SEP client that contains the hotfix after the MS patch?
• How do I tell if I have the correct "Eraser Engine" thing?

Please for the love of $diety, Symantec, consolidate your messaging on this really critical thing into something everybody can understand.  Hopefully somebody can answer my questions above with a clear yes or no.

[John Owens]

@DarrenF-UWD

• With my current version can I install the January patch from MS and have no crashes from the SEP client today?

Yes, but you likely will still have the cosmetic issue explained in TECH248552

• With my current version is the registry setting that the MS patch is looking for - set in the correct manner?

As long as the clients have January 4th definitions or newer.

• With my current version can I install whatever new build of the SEP client that contains the hotfix after the MS patch?

Yes.

• How do I tell if I have the correct "Eraser Engine" thing?

January 4th virus definitions or newer on the client

 

[Appel]

What version is 12.1.5337.5000 ?

[Migration User]

 I have a handful of Winows 10 PCs that are running 12.1.7369.6900 , and just to experiement, I've uninstalled the Windows Security update  KB4056892 and restarted the computers. Seems to correct the issue with false reporting status on the handful of clients that had that security update from Microsoft.

[CanMan]

an old version - see https://support.symantec.com/en_US/article.TECH154475.html

[John Owens]

@Appel

That is SEP 12.1 RU5

[andymease]

I'm hearing the hotfix won't be available until at least end of week

[kahoots1]

Anyone else thinking of going to another AV provider?  

[darcon]

@ kahoots

From what I've seen, all mainstream AV products are having similar issues.  Webroot and Mcafee have build upgrades going out this week to fix the problems much like Symantec.

[Michael Tremmel]

I have over 200 clients running windows 10 build 16299.192 which includes the problomatic windows update patch and I am running SEP 14.0.3876.1100

All of them exibit the same cosmetic issue with the system tray icon but other than that there are no other problems reported. We just need to wait until the new SEP Build is available push it out and stop thinking about it LOL

[CanMan]

There a plety of other things to be doing while waiting - BIOS updates for CVE-2017-5715 are appearing from some vendors. And where appropriate there are Firefox and Chrome updates help mitigate against CVE-2017-5753

[Torb]

I would also advice people to test other software besides SEP. I know that some VPN applications broke because of the MS patch.
Tons of things not related to SEP will also require your attention, so start testing while we wait for a patch.

Pulse Secure typically used by Juniper broke for example.
https://kb.pulsesecure.net/articles/Pulse_Secure_Article/KB43600

 

 

[JASON LINDSEY]

Does anyone know if when Symantec makes the SEP 14 patch available, can it be pushed out from SEPM to Server 2008 R2 and Server 2016 without having to uninstall the current SEP client, reboot, then install the new version?  I only ask because in SEP 12 if you pushed out a new version to servers without uninstalling the old instance first it would create daily error logs in Event Viewer. 

[darcon]

@Jason84

I'm not sure how often you update, but I've never had to do uninstall>reboot>re-install with 12.1 or 14.  Not even when I went from 12.1 to 14.  I usually push out the update via SEPM and check in the morning (servers get restarted @ 3am) to see if there were any problems.  

[John Owens]

We currently have an eta of January 17th for all hotfix patches to be complete. Please remember this is only an estimate.

[S_K]

@Symantec support

Just a quick question regarding the "cosmetic" issue. Do you know if all OS versions are affected or only some (like WIn 10, Windows Server 2016)?

 

[Migration User]

This might help some people. We've decided to patch our development domain server estate. All running SEP 14.0.2349.0100.

The cosmetic issue is impacting all of our Server OSes which consists of the following:

Server 2008 R2

Server 2012 R2

Server 2016 1607

 

The icon issue is something we decided we could live with as we don't have a great deal of users who log on to servers. We preferred to be patched from a Microsoft perspective given we'll be following a complete, staged upgrade process when the Symantec hotfix is released and this takes a little time. 

If you're in a similar position to us, the icon isn't particularly noticeable from a server perspective. We are intending to release the updates to our production domain later if no problems are reported back to us but it's so far, so good. 

[AndrewIT]

hello

i am using sep 14 ru1 mp1

after the windos updates applyed today:

- i have windows 2012 R2 and 2008 R2 server and this problem is only with 2012 R2

- i have the problem with windows 10 also

- no problem with windows 7 and 2008 server

[S_K]

So for now seems Windows 7 machines are not affected, at least what I read here and heard from other people. But good will be if somebody with Win 7 and SEP 12.1 RU6 MP6 and newer write here if he has this tray icon issue

[Migration User]

Thats very slow to fix a potential security issue like this. Leaving PC without Microsoft hotfix is equally risky.

[ℬrίαη]

So until SEP is fixed, no more Windows updates?

https://doublepulsar.com/important-information-about-microsoft-meltdown-cpu-security-fixes-antivirus-vendors-and-you-a852ba0292ec

[darcon]

Only the R2 Servers have gotten the Meltdown/Spectre patch at this time, so Server 2008 and Server 2012 should not be affected by the Symantec issue.

Still on the fence on what to do.  Symantec estimates that the fix will be available in a week, but that very well may change.  I just don't want to be sitting on updates for weeks due to this "cosmetic" issue.  While there hasn't been anything seen using Spectre/Meltdown at this time, you know very well that the bad guys are burning the candle at both ends to exploit these flaws before people get their systems patched.   

[John Owens]

I have not seen this issue (system tray) reported with Windows 7 as of yet.

@Brian

I will check on this. 

[John Owens]

We are currently planning to make hotfixes available for the following releases:

 

• 14 MP2
• 14 RU1
• 14 RU1 MP1
• 12.1 RU6 MP9

[Rudy Garcia]

@John_Owens

Glad you are on the thread.  John helped us before with a major issue we had.

To clarify your last post.  Would we need that version installed to apply the hotfix?  I was assuming the hotfix was going to be a new product update.  For instance we could install lets say "12.1 RU6 MP10" over an older version of 12.1 RU6 MP8

 

I just want to make sure that some of our older version 12 devices do not need to be upgraded to MP9 before installing the hotfix.

[John Owens]

@Rudy Garcia

Thanks Rudy.  No.  For your 12 clients the hotfix will likely be something like SEP 12.1 RU6 MP9a.  You would then just need to grab that full version and upgrade your MP8 clients to MP9a to get the hotfix.

[CanMan]

I only have one Windows 7 machine - it does not have the tray issue. The windows 10 machines do. All SEP 14.1

[Appel]

Will a new software download to upgrade to V14 MP2 contain the fix once this is resolved?

[John Owens]

@Appel

Yes. Once the hotfix is available you would pull down the SEP 14 MP2 hotfix build and it would contain the fix.

[JASON LINDSEY]

@John_Owens

When these hotfix builds are available for SEP 14 would it be recommended to first uninstall the current version, then reboot, and then proceed to push out the hotfix build in regards to versions running on Server 2008 R2 and Server 2016?  Or could it simply be pushed out from SEPM and installed on top of the current version without unistalling first?  Again, I'm only curious in regards to instances running on servers. 

 

Thanks

[[e-mail]]

No need to uninstall, as the version number will be higher, it will be treated as an upgrade of the existing agent.

 

[John Owens]

@Jason84

You will be able to upgrade. No need to uninstall beforehand on Servers.

[hccwloo]

So there will be a patch/upgrade for "Version 14(14.0 MP1) Build 2349 (14.0.2349.0.100)"? When will this be available?

[John Owens]

@hccwloo

 

No.  There will only be a patch/upgrade hotfix for SEP 14 RU1 MP1, SEP 14 RU1, SEP 14 MP2.  You will need to choose one of these versions to upgrade to once the hotfix is available.  Eta is January 17th.

[hccwloo]

Thank you John. We are currently running 14.0 MP1 (Build 2349)

[darcon]

@hccwloo

From what John said, you will need to be running 14.0 MP2, RU1, or RU1 MP1 in order to roll out the hotfix.  This leaves 14.0 MP1 out, so you will need to upgrade to one of the supported versions before you can roll out the fixed version.  

[Migration User]

Hope the fix available soon. Manay of our users get affected.

[Migration User]

Will the hotfix be available for SEPM as well? or is it for clients only?

[jim1292]

Hi John,

Have we got the hot fix ready to resolve this issue ?

 

Thanks.

 

regards,

Jimmy