Incompatible - Symantec SEP v14.3 RU1 and Cisco AnyConnect v4.9.04053 in MacOS Big Sur

[jinwai]

Incompatible - Symantec SEP v14.3 RU1 (14.3.3384.1000) and Cisco AnyConnect v4.9.04053 in MacOS Big Sur

Reported to Symantec.
Anyone else also facing this issue?

[Maurits Sanders]

Hi jinwai, how does this incompatibility show?
We have Anyconnect 4.9.03047 and SEP 14.3.3384.1000 does repeatingly show 'you are at risk' while it can be fixed, but the fix needs to be re-applied very often.

This is on both 10.15.7 and macOS 11.1 Big Sur
We have only a few test devices on Big Sur

[jinwai]

@Maurits Sanders My case only happen in MacOS Big Sur, while working fine for 10.15.7. I keep getting popup asking me to finish setup and "Allow Network Content Filtering".

This user is also facing same issue like me. However, I am not using SEPM, all of my clients are Unmanaged.

https://community.broadcom.com/symantecenterprise/communities/community-home/digestviewer/viewquestion?ContributedContentKey=e91a6ab7-4a88-4462-b30c-43afb15e6f71&CommunityKey=1ecf5f55-9545-44d6-b0f4-4e4a7f5f5e68&tab=digestviewer

[Abbers]

This is interesting because I am using Cisco AnyConnect 4.9.04043 and Symantec SEP v14.3 RU1 (14.3.3384.1000).

So that's at least three of us with the same problem.

I will install a clean MacOS Big Sur with unmanaged SEP v14.3 RU1 only (I will not install Cisco AnyConnect) and I'll see if the problem occurs.

Thank you

[Abbers]

So far, so good.

I've reinstalled a clean MacOS Big Sur with unmanaged SEP v14.3 RU1 (14.3.3384.1000) and I've not seen any popup warnings for almost two hours.

I have NOT yet installed Cisco AnyConnect.

[Abbers]

So my installation of SEP v14.3 RU1 (14.3.3384.1000) worked fine for several hours until I installed Cisco Anyconnect.

Specifically, it was when I enabled the "Cisco AnyConnect Socket Filter" during AnyConnect installation that SEP began interrupting me every ten minutes:

Are we seeing the effect of a conflict between Cisco AnyConnect's Socket Filter and Symantec Endpoint Protection's Network Content Filter ?

[jinwai]

Yes, it is conflict and incompatibility between SEP and AnyConnect inside Big Sur, like my title in this thread.

[Abbers]

Have you reported this issue to Cisco ?

You might get a quicker response from Cisco as they are pushing out AnyConnect updates on a regular basis.

 

I've tried with the latest Cisco AnyConnect 4.9.05042 (released last week) and the problem still occurs.

[jinwai]

Hi Abbers, yes I saw you found my post in Cisco forum below. Thank you very much for your detail explanation and screenshots in the forum.

https://community.cisco.com/t5/vpn/incompatible-symantec-sep-v14-3-ru1-and-cisco-anyconnect-v4-9/m-p/4258585

[Maurits Sanders]

Anybody found a fix?

[Jeremy Morales]

I was at this all day yesterday. I attempted to disable just the Cisco filter extension based off information here:

https://www.cisco.com/c/en/us/td/docs/security/vpn_client/anyconnect/anyconnect49/upgrade/AnyConnect_macOS_BigSur_Advisory.html#_Toc52277817

but after removing it, as soon as I rebooted, Cisco was asking for their extension to be approved. I denied it, but SEP was still unable to resolve itself.

[Abbers]

Thank you Jeremy Morales. Your dedication in trying to resolve the issue is appreciated, and has inspired me to take matters into my hands to find an unofficial solution.

Short answer:
Removing the "Cisco AnyConnect Socket Filter" application from the \Applications\Cisco AnyConnect folder stops the SEP warning from appearing.  SEP and Cisco AnyConnect continue to function correctly

Long answer:

Attempt 1)
I tried disabling all three Cisco AnyConnect Socket Filters that were present in System Preferences -> Network, but that did not fix the problem. The SEP warning prompt kept reappearing.

Attempt 2)
So what I did instead, was to reinstall the Cisco AnyConnect client, but this time I did NOT allow the Cisco AnyConnect Socket Filter to run.
When the AnyConnect installer prompted me to enable the Socket Filter, I ignored the prompt, and I let the AnyConnect installation continue.

There were no Cisco AnyConnect Socket Filters present under System Preferences -> Network, and my SEP installation did not displayed the warning prompt for over 30 minutes (it never managed more than 10 minutes before).

Cisco AnyConnect VPN client still worked.

It all seemed to work OK . . . until I rebooted.

After the reboot, the Cisco AnyConnect Socket Filter kept prompting me until I gave in and enabled it . . . and then immediately the SEP warning appeared.

Attempt 3)
I then removed (not just disabled) the three Cisco AnyConnect Socket Filters that were present in System Preferences -> Network, and the warning screen stopped appearing . . . until I rebooted.

Attempt 4)
What I did this time, was manually delete the "Cisco AnyConnect Socket Filter" from the \Applications\Cisco AnyConnect folder . . . yes, very dangerous, I know, but trust me, I'm a professional.

MacOS displayed a warning that the application "Cisco AnyConnect Socket Filter" is hosting System Extensions. These extensions will be removed if you continue.

I accepted the warning, and rebooted . . . so far, the Cisco AnyConnect client works, and SEP still works and has not displayed the warning.

Now, it's approaching midnight here in Europe, so I'll leave it there for tonight. I'll add screenshots to this post in the morning, because everybody likes pictures, and I'll proofread it properly, because nobody likes spelling mistakes.

I'll update the article on Cisco's support forum tomorrow morning too.

Perhaps Cisco technical support's reaction to me removing part of their AnyConnect software will push Cisco and Symantec to fix this issue properly.

Try this at your own risk, ideally on a test Mac first.

[Jeremy Morales]

Thank you, Abbers, I'm testing that out on my test Mac now to see if just directly deleting it causes any issues besides just deleting that extension.

[Jeremy Morales]

And this didnt work for me. Extension is deleted, computer is rebooted, multiple times, still shows "Fix" and "At risk" maybe you got lucky

[Abbers]

Do you have any Cisco AnyConnect Socket Filter entries in System Preferences -> Network?  I don't have any.

It may have been the order in which I did the steps:

Disable the three Socket filters then reboot
Delete the three Socket filters then reboot
Drag Cisco AnyConnect Socket Filter to the trash and accept the warnings then reboot

[Jeremy Morales]

Nope, no extensions in Network and there was only the Content filter in the Application\Cisco folder.

[Abbers]

Do you mean Socket Filter, in the \Applications\Cisco folder?

I do not see a Content Filter app in that folder

Here's the warning I got when I manually removed the Socket Filter app​

It was a screenshot on a Retina display, so it's "huge" but you can see that I only moved Cisco AnyConnect Socket Filter to the trash (and I haven't even deleted it from the trash yet - it's still in there).

AnyConnect and SEP are happily working together after several reboots.

I'm using AnyConnect 4.9.04053, and I installed the AnyConnect VPN client only - not the additional components such as Posture, Umbrella etc

[Jeremy Morales]

So yes, that was the extension I deleted. 

We do run Umbrella client as well, which handles our DNS redirect, obviously, so I wonder if that has anything to do with it.

[Abbers]

I will test the AnyConnect VPN client install on a second Big Sur test Mac, and I'll report back with my results.

[jinwai]

Hi,
Thank you both of you for doing great testing.
Have you created case for this issue with Broadcom and Cisco? I have created case few weeks ago and only receive 1 reply from each of them, quite slow response and solution from them.

[Abbers]

I installed Cisco AnyConnect 4.9.04043 and 4.9.04053 on my second Big Sur test Mac, and have had no problems with either of my two Macs since last night.

What I did do, as part of the test, was to uninstall then reinstall Cisco AnyConnect again on each Mac, and fully complete the installation process (including allowing the system extensions).

Maybe the AnyConnect reinstall fixed the problem (and I reinstalled AnyConnect several times when testing), but I've had AnyConnect and SEP running for several hours, without any conflicts.

Here's a screenshot of my primary Mac.  Again, please excuse the image, it was taken on a Retina display and I wanted to show as much information as possible:

I have a functioning Symantec Endpoint 14.3 RU1 on the left, with Cisco AnyConnect 4.9.04053 on the right.

As a more comprehensive test, I installed AnyConnect with every available option selected, and I did not disable or remove any of the Cisco AnyConnect Socket Filters from System Preferences -> Network, and I did not delete AnyConnect files from the the file system.

AnyConnect and SEP have been running side by side all day, with no conflicts reported.

Even after rebooting, AnyConnect and SEP do not conflict with each other.

The only change I have made since last night was to reinstall AnyConnect during testing.

[jinwai]

Great to hear that you have found a solution. I wonder would it work if you only install / reinstall AnyConnect with VPN option only (excluding all other AnyConnect options)​?
Today Cisco support replied my case, at least they are responding faster than Broadcom. I have asked them to refer to this discussion.

[Abbers]

Reinstalling with the VPN only works for me.  That is how I usually configure my Cisco AnyConnect client.

I only reinstalled AnyConnect with all options so that I could test Jeremy Morales' setup with Umbrella installed.

Here is AnyConnect v4.9.04053 VPN only, working alongside SEP 14.3 RU1 on my secondary test Mac earlier today.

I have rebooted the test Mac, and I still have no conflicts:

[Jeremy Morales]

Okay, I may have resolved it too, after reading your guys post.

1. Uninstall both Symantec AND Cisco AnyConnect
2. Reboot
3. Install Cisco AnyConnect, complete setup completely, including allowing system extension in Security/Privacy
4. Reboot.
5. Install SEP, approve system extension in Security/Privacy, and also making sure Full Disk was allowed (which it was in my case)
6. Reboot.

After this final reboot, it appears to be holding good. 4 minutes so far and no "Fix it" pop up. Usually this was pretty immediate.

[Jeremy Morales]

......and 22 minutes later, it went back to "Fix it"

[Abbers]

Jeremy, can you try uninstalling and reinstalling Cisco AnyConnect, and enabling the system extension.

[Jeremy Morales]

Nope, that didnt work. I'm going to uninstall Umbrella next as its a "filter" as well.

[Abbers]

What AnyConnect version are you using ?

[Jeremy Morales]

4.9.04043 is my version. I can't upgrade as it would lose communication to our VPN server and that is controlled by another team, nor can we push out a newer version of the VPN client at this time, because it would go to everyone.

Uninstalling umbrella results in the same, "Fix it".

At least it is communicating to my SEPM server and definitions are up to date.

[John Owens]

Here is the document for this issue Broadcom is investigating. https://knowledge.broadcom.com/external/article/206091

Please continue to monitor it as it will be updated once workarounds or a fix is available.

[jinwai]

Finally a Broadcom staff reply with a good article specifically for this issue.
Since there is no option to subscribe alert for that page, I am using https://visualping.io/ to automatically monitor for me.
Hope there will be a fix very soon. Thank you everyone's effort, especially Abbers.

[Abbers]

I did some further testing last night to find other other possible workarounds, but the only one that consistently worked was to remove the "Cisco AnyConnect Socket Filter" application from the Applications\Cisco folder.

This action removes the system extensions from System Preferences -> Network, and allows SEP to run without displaying the warning every ten minutes.

I understand this workaround may only be suitable for organizations using the basic VPN client, and not those who require additional AnyConnect components such as Umbrella, Posture and Web Security.

I tested using the following Cisco AnyConnect versions, and my results were the same in all cases:

4.9.04043
4.9.04053
4.9.05042

What didn't work
1) AnyConnect reinstall: My earlier recommendation of reinstalling the Cisco AnyConnect client did NOT prevent the warning from appearing.

2) Disabling the Cisco AnyConnect Socket Filters from System Preferences -> Network did not fix the problem:

3) Deleting the Cisco AnyConnect Socket Filters from System Preferences -> Network did temporarily prevent the warning from appearing, until the next reboot.
After rebooting, the warning would appear again, and the three Cisco AnyConnect Socket Filters would reappear in System Preferences -> Network and be enabled.

Has anybody else tried the workaround of removing the Cisco AnyConnect Socket Filter from \Applications\Cisco ?  It's worked for me on my two Big Sur test Macs, but I'd appreciate other feedback because I will start applying the workaround to live Macs from next week.

Thank you

[jinwai]

I have received followup email from Cisco below. I think our option is to wait Broadcom / Symantec to fix it, however, their response is very slow, and no follow up email from them. 

======
I apologize for any inconvenience caused. Let me rephrase what I had informed earlier in my email.

It is not just by Symantec update we are saying the issue is not on our end. The two (2) Socket Filters should be independent of one another. Please note that the behaviour by SEP could very well occur with any other 3rd party application that installs a Socket Filter. (not suggesting this is the case, but just pointing out that Socket Filters are used by many other applications as well).
The above being said, SEP, by nature being an AntiMalware product, primary responsibility is to try to determine if there are anomalies on endpoints and/or unexpected behaviour of certain files, actions, etc, etc. It seems that in this case, which is supported by their own article, that SEP is responsible for the incompatibility….moreover, which starts by falsely claiming their setup has not finished (the You are at risk! Pop up)… even though the User(s) – we assume and they/Symantec assume - clearly have subsequently clicked on their (Symantec’s) Allow Network Content Filtering pop up.. which should have completed the SEP setup…but apparently does not, where SEP subsequently and repeatedly pops up their ‘You are at risk!’ indicating their own setup is not complete.

If this was AnyConnect issue as well we would have received multiple calls by now which is not the case at the moment.

The fact that the AnyConnect socket filter might be resolving the issue is because the Socket Filter is no longer active; which however does *not simply imply that AnyConnect is root cause. Again, SEP is the one determining (and falsely determining) their setup is not complete, even though there is all indication that the User(s) successfully completed the SEP setup.
Another way to look at this is AnyConnect does not interact or hook into SEP in anyway; therefore, SEP can only be solely responsible for why their own product is claiming their setup has not completed.

In my previous email I requested for Symantec installer file and not the MAC device as we will not have access to all 3rd party applications as the list as the list of 3rd party products is MASSIVE and always changing. More importantly, it is unrealistic for any product team and/or even 3rd party product teams/applications/vendors to maintain test beds in which continuously test for potential compatibility issues amongst others products. It is simply not feasible.

If you still wish for us to replicate, you will have to provide us with the SEP setup file/key and involve the Symantec support as it critical for them to be involved as the pop-up is from their software claiming setup is not complete.

We have updated the same in our cisco form. Below is the link for the same.

https://community.cisco.com/t5/vpn/incompatible-symantec-sep-v14-3-ru1-and-cisco-anyconnect-v4-9/td-p/4258585
======

[John Owens]

Good Morning Everyone,

There is a fix available for this issue. Please open a support case if you are experiencing this specific issue and request.

Thanks!

[jinwai]

@John_Owens
When will be the expected next release version for SEP that will include this hotfix? We prefer to have stable release instead of using hotfix. Thanks.​​​

[John Owens]

I believe 14.3 RU1 MP1 will have the fix as well. ETA is March/April for this build.

[sysadmin thevalley]

So how does this work for the SES (cloud version)
RU1 is supposed to be available now (end of januari) but when it's released this will also have this problem / how can a hotfix be applied ?

[John Owens]

SES 14.3 RU1 (once released) will have this fix merged into it. So nothing will need to be done.  Thanks!

[Matt Ohm]

Their fix is to select don't allow and wait for the next version.  Save yourself the headache of calling support. 

[John Owens]

What was your case number. I will assist here.

[Matt Ohm]

32513808

[John Owens]

Please check your email.

[jinwai]

Hi everyone. They have released a fixed in version RU1 MP1. The problem has been fixed  

https://knowledge.broadcom.com/external/article/206091