dramatic LAN speed change after installing Endpoint Protection 11

[Migration User]

We have a Gigabit LAN, and copy speed on Windows XP machines was about 50000 60000 kb/sec.
After installing Endpoint Protection 11 it went down to 30000-35000 kb/sec.
On server where system is Windows 2003 advanced server, and is installed protection manager+client its even worse 10000-12000 kb/sec.

We tryed to uninstall endpoint and everything went back to normal speeds.
Please help with this problem, maybe there is some kind of setting that is killing LAN copy speed?

I understand there must be minor speed change but not 1/2...

Thanks in advance.

[Fatih Teke]

Hello
what is your cliends settings? Did you open network scan? and are all clients definitions time same?

Regards.
Fatih

[Migration User]

Hi
Settings are default, havent change anything.
Definitions are same. All are updating from SEPM without problem.

[Fatih Teke]

Hello again.
Can you try this please,
In the SEP Manager >Policies>Antivirus Policy (which used) edit > File System Auto Protect > Network Settings >Trust Files on Remote Computers running auto protect check box (choice)

Regards.
Fatih

[Migration User]

Just tryed, no difference.

[Fatih Teke]

What is your client connection type? and can you sniff lan via wireshark?
As you well know all Antivirus programs scan folders when copying. but i am not sure your copy is too slow.

Best Regards.
Fatih

[Migration User]

Well as i stated above, i know that AV software slow everything down. But i just want to know is that DRAMATIC speed decrease normal, or i can tweak something to make it faster. I have full Gigabit LAN, which goest up to 60 mb/sec without EP.

Wireshark works well too, though i prefer CommView.

[Fatih Teke]

Hello.
Maybe this article will help you.
https://www-secure.symantec.com/connect/articles/how-much-bandwidth-used-sep-client-one-day

Best Regards.
Fatih

[Migration User]

Btw, Linkspeed utility from Microsoft shows, i have solid 953 mbps link between all my computers.

[Migration User]

M, i just found out that disabling Network Threat Protection module of SEP, giving speed boost...

[Migration User]

Any Symantec Employee, or Trusted Advisor, can help?
plz

[Migration User]

Here is a guide on how to optimize the performance of the SEP clients. Some of the changes described can decrease traffic on your network which could help. 

http://service1.symantec.com/SUPPORT/ent-security.nsf/docid/2007102311173048

Bottom line is that anytime you are reading in and scanning packets there is going to be a overhead. However your overhead seems to be more dramatic than I have seen. If the changes described in the article above don't help I would call in for support. They will be able to look into your network more easily than we can via the forums and maybe provide even better alternatives.

Hope this helps,
Grant

[Migration User]

This didnt help :(
However, after removing Network Threat protection im receiving nice LAN speed boost, about 5/mb sec.

[ℬrίαη]

If you want to keep NTP on your clients in order to use certain features, you can withdraw the firewall policy as a test and see what happens to your LAN speed.

[Migration User]

How exactly can i withdraw firewall policy?

[Fatih Teke]

go to your clients. choce second tab. you will see yuor policies. (by the way uncheck inheritance policy)
choice your firewall policy and click withdraw in left side.

Regards.
Fatih

[ℬrίαη]

You can also navigate to Policies tab, select the firewall policy in use for the specific location and select Withdraw the Policy on the bottom left hand side.

[Migration User]

That didnt help :(
Guess im going to create new install packages without Network Threat Protection

One question about that. When i deploy new packages, my windows firewall get enabled. How i can fix that? I dont need it.

[Migration User]

on my Windows 7 64-bit Dell computer. Had to downgrade the Intel LAN driver from Dell to MS version. Now it works ok!

[Fatih Teke]

maybe it works for emperor.
can your try update network card driver please?

Regards.
Fatih

[Migration User]

Can anyone answer me, why i get normal LAN speed after uninstalling NTP module, but getting slowdown when i just withdraw firewall policy ?

[Migration User]

Have you read the help file or the Administrators guide to understand what NTP is exactly, what make up "Network Threat Protection," and how it protects your computers.  Your answer lies within.

That said, I have never seen that much of a change in file copy tests except for heavily loaded servers that already average 30-50% of used bandwidth during production hours.  For desktops, it's been negligible at best.  Of course, you may have better luck as suggested with different driver versions, whether from the manufacturer itself (Intel, Broadcom, etc) Microsoft's own generic driver, or latest and greatest OE vendor drivers from the system maker.

You may also want to make sure you are on RU5 or RU6.

[Migration User]

I wish I had looked here for help a couple of weeks ago. In our situation, we introduced Cisco WAAS (Wide Area Application Services) to several remote locations. Everything was running fine until we centralized their file servers. Once that happened, clients were getting 30-45 second delays in opening and saving files across the WAN. We engaged Cisco TAC for over 3 weeks with no luck. Finally a co-worked stumbled across the Network setting mentioned in this thread and the KB referenced above. It made a HUGE difference.

Prior to finding the fix we sniffed the Cisco WAAS appliance using TCPDUMP and Wireshark. We saw many data transfers that would start and stop between the file server and the WAAS appliance. After un-checking the Network option in File System Auto Protect we started getting clean packet captures.

[thromada]

I have experienced the EXACT same issue with having NTP enabled (https://www-secure.symantec.com/connect/forums/network-threat-protection-always-slows-down-true).  We want to use NTP so we can use Application and Device Control.  But throughput is GREATLY affected on the clients.

[Migration User]

I still wait some official comments about that.
If this slowdown is INTENDED, that i have nothing to say, and its just up to me to decide whether live NTP enabled or disable it.]
But, if there is some way to get speed up, without removing NTP - im all ears..

[Migration User]

Side note, are you using App and Device Control with NTP?  There is a known issue with slowdowns in this particular config.  
If you had experienced this, and removed App and Device control, the speed would come back to normal..

There is a hotfix in the wings for this I believe.

[Migration User]

I think you should take a look at the Admin guide and read up on what NTP is and does. I think if you better understood how NTP works then you would understand the slowdown. You could also look at this forum thread which is very similar https://www-secure.symantec.com/connect/forums/network-threat-protection-always-slows-down-true. Many Symantec employees have commented in this thread. But again like I said before you have to expect that there will be SOME slowdown in the client network due to  how the Teefer driver sniffs and examines the packets being sent around.

If you want the official "Symantec" view then please consult my post before I have already given it:

"Bottom line is that anytime you are reading in and scanning packets there is going to be a overhead. However your overhead seems to be more dramatic than I have seen. If the changes described in the article above don't help I would call in for support. They will be able to look into your network more easily than we can via the forums and maybe provide even better alternatives."

If you are still having troubles or want a better more complete explanation then call support. You should have a support contract if you are running the software.

Grant-

[thromada]

There's been other replies on this subject of slow througput that similarly say something like "if you read about NTP and firewalls, you'll understand about the slowness".  But look at this dramatic difference from my other post that's been sited here:

As a test, I happen to be copying 1,289 items at a size of 89MB from one client hard drive to another on the same subnet.  With NTP enabled it took 1min 44sec.  With NTP disabled it took 31sec.  That's dramatic and unsupportable.

I don't think it's the NIC drivers or the OS's (Win XP Pro SP3) or the versions - it's the same results with RU5 and RU6.  Disabling or withdraing the Firewall policy has no effect.  It's verified and repeat-able.  Disable a client's NTP from the SEPM console and you're througput is back to normal.  Are god-emperor and I the only ones observing this issue?

Would somebody else simply please take a few minutes and do a similar test and post the results?  Thanks.

[Migration User]

In your NTP (Firewall policy) do you have "Enable reverse DNS Lookup" Enabled?  Try disabling it and see what happens.  The firewall rules (and logs) will constantly be doing PTR lookups on your DNS server.  That just adds latency between operations, resulting in slower performance overall.  The actual packet is small.. but the time between the results is much greater.

However, if you disable this, then you cannot use DNS names in your firewall rules.

Check that out.

-Pete Sutsos
Systems Administrator, K-Swiss, Inc.

[Migration User]

[thromada]

Thanks Pete Sutsos, but "Enable reverse DNS lookup" is disabled.  In fact, the entire Firewall policy is disabled.  NTP is only installed in order to use Application and Device Control.  The only way to see throughput go back to a normal speed is to disable (or uninstall) NTP.

At this point, I just want to know if anybody else can do a similar test and see if the results are the same.

[Migration User]

Any update about this issue?

[Migration User]

There was such a problem with SEP MR4 MP1 and MP2 but it fixed in SEP MR5. have you tried upgrading all your clients to SEP MR5?

[Migration User]

I too am having this issue.  I just posted this in another SEP Thread https://www-secure.symantec.com/connect/forums/network-threat-protection-always-slows-down-true#comment-3981371

I'm also experiencing slow network performance on RU6.  Now i do understanding that i would see some impact with NTP enabled, but not as much as what I'm recording.  I setup 2 Windows XP Pro VM machines.  Copied a 500MB file from a network location to the VM machines.  I then uninstalled NTP on one of the machines, and disabled Bloodhound detection on the other one (a possible solution from another thread) and copied a different 500MB file.  Results below

	NTP Installed	NTP Uninstalled	Bloodhound Disabled
Test Machine 1	50.2 seconds		45 seconds
Test Machines 2	46.2 seconds	23.9 seconds

Now thats a BIG difference with NTP uninstalled.  We use NTP so monitor and block removable USB media.  Its now getting to the stage where i will have to look for another product to do this and leave SEP just for AV

[Migration User]

I'm guessing that this has been placed in the too hard basket?

As I'm getting more and more complains about network speed i did another quick test.  I used 1 machine with NTP installed.  After transferring 1 file i installed NTP and rebooted.  I then copied the same file and a different file roughly the same size.  Results speak for themselves.

	NTP	NTP Disabled
File 1 (569 MB)	52 seconds	28 seconds
File 2 (572MB)		27 seconds

   
 

[thromada]

Yes this is a problem.   Nor have I opened a ticket with Symantec support.  The replies here have been few to none with those experiencing this issue.  Or this issue has been discounted as just what it means to have a firewall installed.  But with NTP enabled (and Firewall policy disabled - I'm not using the Firewall rules), normal computer use (MS Office, other applications) seems unaffected.  But if copying large amounts of data (big files or many files), that's when it's noticable.  I haven't had any users complain about speed yet during normal computer usage.  I wanted to see as I install SEP RU6 on clients if they notice a problem with their normal computer use.

[Migration User]

Guys who use SEP 11.0.6, can anyone confirm this issue still not fixed? i have 11.0.5, and wondering if upgrading can give me something..

[thromada]

I'm running 11.0.6 and ran another test just to be sure; and the issue remains.  Copy 1279 files at 96MB on desk with NTP enabled = 1min 14sec.  With NTP disabled = 30sec.