DX NetOps

 View Only

Spec KB:  Third party applications can exhaust Windows kernel memory and re

  • 1.  Spec KB:  Third party applications can exhaust Windows kernel memory and re

    Broadcom Employee
    Posted Jan 06, 2014 09:04 AM

    Document ID:    TEC599290

     

    Spec KB:  Third party applications can exhaust Windows kernel memory and result in Spectrum performance problems and crashes

     

    Description:

    Windows operating systems use two finite memory pools for kernel components, based on the amount of physical memory on the machine and the architecture of the Windows version. These are the paged pool, which can be paged out to disk if physical memory is low, and the much smaller non-paged pool (NPP), which guarantees that memory allocated will remain in physical memory. Once non-paged memory has been allocated it cannot be freed. For more information on the Windows kernel address space, see the references below.

    What does this have to do with Spectrum? The SpectroSERVER and Archive Manager can be indirectly impacted when non-paged pool memory is exhausted. Third party applications (often antivirus, backup and Host Intrusion Prevention (or HIPS) applications) can consume all available memory in the non-paged pool (NPP). When antivirus and host intrusion applications scan the system, they cache data in the NPP. When most of the non-paged pool memory has been allocated, the Windows kernel (srv.exe process) can't allocate more for itself or for application handles and drivers. This can result in SpectroSERVER and Archive Manager crashes, as well as performance problems in the TCPIP stack.

    Some examples (not an exhaustive list) of applications that could consume kernel memory:

         Veritas
          ARCserve Backup Agent for Open Files or ARCserve Open File Agent
          Inoculan
          McAfee antivirus and/or Host Intrusion Prevention System
          Trend Micro
          Symantec antivirus

    SYMPTOMS:

    Remote desktops stop responding. The system is slow overall.

    The Windows Application event log might include these events:

         Event ID: 2022
          Source: Srv
          Type: Error
          Description:
          Server was unable to find a free connection <number> times in the last <number> seconds.

         Event ID: 2021
          Source: Srv
          Type: Error
          Description:
          Server was unable to create a work item <number>> times in the last <number> seconds.

         Event ID: 2019
          Source: Srv
          Type: Error
          Description:
          The server was unable to allocate from the system nonpaged pool because the pool was empty.

    It is also possible for third party applications to exhaust the paged pool, in which case you will see this event:

         Event ID: 2020
          Source: Srv
          Type: Error
          Description:
          The server was unable to allocate from the system paged pool because the pool was empty.

    The Spectrum $SPECROOT/SS/VNM.OUT file and Spectrum Control Panel could show this message: "Fatal error unable to allocate heap"

    The problem is more likely to occur on pre-Vista and 32-bit pre Windows Server 2008 versions of Windows. The problem also might start happening after adding the /3GB switch to the boot.ini. This cuts the available kernel memory in half (see the technet blog).

    Although there are known non-paged pool leaks in some antivirus and Host Intrusion Prevention software releases, it is the overall allocation of non-paged pool memory that causes the problem. In other words, a leak could be responsible, but your third party software might be working as designed and still exhaust all available NPP memory.

    Please select the following link for more information on this subject:

    Knowledge Document Link:

    https://comm.support.ca.com/?legacyid=TEC599290