Endpoint Protection

Securing Windows 2000 Communications with IP Security Filters, Part 1 

03-21-2002 02:00 AM

by Joe Klemencic

Securing Windows 2000 Communications with IP Security Filters, Part One
by Joe Klemencic
last updated March 21, 2002

With the release of Windows 2000, a new feature, IP Security, was added to allow for more granular control of IP-based traffic over the previous Windows NT4 packet filter option, TCP/IP Filtering. Originally, when the TCP/IP Filtering option was enabled, it was applied to all network adapters on the host system and could only affect the protocol used. For example, there was no provision to allow NetBIOS only from select hosts while allowing HTTP from any host.

The main premise behind the TCP/IP Filters is to allow specialized server configurations to be generically secured for only their intended traffic. Since NetBIOS cannot easily be disabled on a Windows NT4 server, one would implement TCP/IP Filters on their IIS installation to allow only HTTP traffic to the server and block all other types of traffic. The original TCP/IP Filters implementation only inspected inbound traffic originating from outside the host to be inspected. All traffic originating from the host was not inspected by the TCP/IP Filters.

The new IP Security feature included with Windows 2000 expands greatly on the original TCP/IP Filter, even though the legacy packet filter is still available for use in Windows 2000. The IP Security feature includes the ability to create specific traffic filters, specify the source and destination addresses, specify the protocol and service, inspect both inbound and outbound traffic, and encrypt the data streams using IPSEC. The original idea of the IP Security feature in Windows 2000 was to provide secure communications using IPSEC between hosts and services. An IP Security policy contains various filters and filter actions and, optionally, an authentication and encryption scheme. IP Security filters also have the ability to create a ‘tunnel’ between two defined endpoints.

Soon after the public beta release of Windows 2000, beta testers found a way to implement the Windows 2000 IP Security filters as a flexible packet filter system. This new usage method allows a host to selectively permit and deny traffic, much like personal firewall applications. Microsoft soon adopted support for this new use outside of their original intention by publishing usage documents. However, to this day, tools to monitor and troubleshoot such an implementation are still non-existent.

This article is the first of a two-part series that will describe the various methods of implementing Windows 2000 IP Security filters that are integrated with IPSEC communications. The series will attempt to describe the function of the features available, how to configure them and how to troubleshoot the installations. It will conclude with recommendations of how to implement each type of IP Security configuration in different scenarios. This article will offer an overview of IP security policies, including defining, testing, and expanding IP security policies.

Overview of IP Security Policies

Windows 2000 IP Security filters are defined (see figure 1) in the "IP Security Policies on Local Machine" view located under the "Computer Configuration Security Settings" in the Group Policy Editor snap-in for the Microsoft Management Console (MMC) application. Alternately, this snap-in can be launched directly by executing the GPEDIT.MSC application from the START menu.

Fig 1: Snapshot of the default IP Security policies

Three IP Security policies are defined by default:

  • Client (Respond Only)
  • Require Security
  • Request Security

Client (Respond Only)

This policy is typically used by workstations to respond to authorization requests from Windows 2000 servers that require a secure authentication before using a service.

Secure Server (Require Security)

This policy is used on Windows 2000 servers and Windows 2000 hosts that provide network-based services to ensure that non-authorized or non-encrypted traffic is denied from communication. This setting only works in a pure Windows 2000 environment.

Server (Request Security)

This policy is used on Windows 2000 services in the same fashion as the Secure Server policy, except this policy attempts to negotiate the highest level of authentication and encryption possible with the client. This setting is useful in a mixed-mode environment. As stated previously, the possibility exists to create custom policies for use on a Windows 2000 installation.

Defining a Custom IP Security Policy

By right-clicking in the "Group Policy" policies window, you will be presented with menus allowing the creation of a new IP Security policy and the ability to manage IP filter lists and actions. In this walkthrough, we will start by creating a basic filter action with a BLOCK action, then will create a filter set that will allow Windows Terminal Services from a specific set of IP addresses and deny it from any other address.

Create the DENY Filter Action

By selecting the "Manage IP Filter Lists" and "Filter Actions" from the right-click menu, you will be presented with the interface to create new filters and filter actions (Fig 2).

Fig 2: Manage IP Filter Lists and Filter Actions

Select the "Manage Filter Actions" tab and click the "Add" button. Under the "Security Methods" tab, you will be presented with three radio buttons (Fig 3): "Permit", "Block" and "Negotiate Security". Since we are defining a "DENY" filter action, select the "BLOCK" radio button. Now, define a name to this BLOCK operation so it can easily be referred to in a filter. Select the "General" tab and enter "DENY" in the Name field. Optionally, a description may be given to describe the intent of this new filter action. Click "OK" when finished. You should now see the new filter action named "DENY" in the "Filter Actions" window.

Fig 3: New Filter Action Properties

Create the Restrictive Filters

Click on the "Manage IP Filter Lists" tab and click the "Add" button. You will now be presented with the "New IP Filter List" window. Set the Name field to "Allow Terminal Services" with a brief description of this filter. Ensure that the "Use Add Wizard" checkbox is selected and click the "Add" button. You should now be presented with the "Filter Properties Wizard". After clicking on the "Next" button, you should be prompted to enter the IP Traffic Source address. Five different choices are available for this:

  • My IP Address - Specifies that the filter will use the current IP address assigned to the system. This setting is recommended on a host that contains a single network connection that uses either DHCP or has a static IP address, and does not perform routing functions. Using this setting is not recommended when multiple network interfaces are installed and active on a host
  • Any IP Address - Specifies that any source IP address will be evaluated.
  • A Specific DNS Name - Specifies that the host is defined by a DNS name. Since the IP Security filters utilize IP addresses, this selection will perform a DNS lookup of the supplied host-name and will insert the actual IP address into the filter rule during the filter creation.
  • A Specific IP Address - Specifies that a single IP address is to be utilized.
  • A Specific IP Subnet - Specifies that a network range will be used by entering the IP subnet network and the associated netmask.

Since we are defining a filter that will restrict Terminal Services from a specific subnet, select the "Specific IP Subnet" item. Enter the network IP address and the associated netmask in the boxes provided, and click the "Next" button. In our examples, we will assume that our company's subnet is with a netmask of To permit "Terminal Services" from our entire network, enter as the IP Address and as the subnet mask.

Next, you will be presented with the "IP Traffic Destination" dialog box. The options for the selection are the same as the "IP Traffic Source" dialog box. Since we are creating a filter that will restrict terminal services to this local server, select "My IP Address" and click "Next".

Next, you will be prompted to select the protocol type to be used in this filter. Many different protocols are defined. Since Terminal Services utilizes TCP connectivity, select the "TCP" option and click the "Next" button.

You will now need to select the port to be inspected in this filter. Since Terminal Services utilizes port 3389, and we are defining the rule to allow Terminal Services to the local machine, select the "To This Port" radio button, enter "3389" in the text field, and click the "Next" button. Click the "Finished" button when done.

The filter that was just created will allow Terminal Services from the specified subnet to the local host; but since it is simply a packet filter, a reverse rule must also be created. Microsoft includes the Mirror Rule option, but it has been my experience that when using the IP filters to control traffic it does not always apply as desired. To ensure that no problems arise, we will manually create the reverse rule. Once again, select the "Add" button and enter the following in the appropriate "IP Filter Wizards" dialog boxes:

  • Source Address: My IP Address
  • Destination Address: A Specific IP Subnet
  • Destination IP Address:
  • Destination Subnet Mask:
  • Protocol Type: TCP
  • From This Port: 3389

Ensure that you enter these in the correct locations, especially the "From This Port" entry. This is important because all responses from the local host's Terminal Server application will be sent back to the clients with a source port of 3389. Figure 4 contains the completed IP filters for comparison.

Fig 4: Completed Allow Terminal Services filters

We now need to define the filter that will block Terminal Services from unintended hosts. Following the steps outlined above, create a new IP Filter List named "Deny Terminal Services" with the following filter. Figure 5 contains this new IP Filter.

  • Source Address: Any IP Address
  • Desination Address: My IP Address
  • Protocol Type: TCP
  • To This Port: 3389

Fig 5: Completed Deny Terminal Services filter

Before we proceed, you may notice that these filters overlap in their definition. The IP Security policies do not have a user-defined ordering for defined filters. Instead, the filters are evaluated from the most granular to the least specific. Since we created an "Allow Terminal Services" filter that specifies both a source subnet AND an explicit destination, it will be evaluated before the more general filter of any source address to an explicit IP address. Typically, the filters are evaluated in the following orders:

  • My IP Address
  • Specific IP Address defined
  • Specific IP Subnet
  • Any IP Address

Consequently, the protocols and services are also evaluated in a similar fashion:

  • Specific Protocol/Port combination
  • Specific Protocol/Any Port
  • Any Protocol

At this point, all we need to do is put it all together in an IP Security policy. In order to do so, implement the following steps:

  1. Go back to the Group Policy window and right-click in the policies window pane and select "Create IP Security Policy".
  2. Click next through the IP Security Policy Wizard and assign a name and a description to this new policy and click "Next".
  3. You should be prompted to use the default response rule. Since this policy will only be inspecting packets destined for Terminal Services and will not be providing authentication or encryption, uncheck the default response rule box and click "Next".
  4. Ensure that the "Edit Properties" box is checked and click the "Finish" button.
  5. At this point, you should be presented with a properties window for this new IP Security policy in which you can now add the filters you created previously. To make it easier to follow the first time through, ensure that the "Use Wizard" box is checked and click the "Add" button.
  6. The first decision you need to make is if this filter policy will use a tunnel. Since we are not tunneling traffic but simply allowing or denying traffic, select the "This Rule Does Not Specify a Tunnel" radio button and click the "Next" button.
  7. Unlike the previous Windows NT4 TCP/IP Filters, the Windows 2000 IP Security filters allows you to choose on what interfaces to apply the filter. Since you only have one network card installed and active, select the "All Network Connections" radio button. This will also ensure that if another network connection is established in the future, these filters will still apply.
  8. Next, you will be prompted for an authentication scheme to use for this filter. Once again, since we are only allowing and denying packets, keep the default Windows 2000 default selected and click the "Next" button.

On the next screen, you should see the filters you created (Fig 6). Through these next steps, we will be linking the appropriate filter action to their associated filter.

Fig 6: Defined IP Filters List

  1. The next screen lists the available filter actions that can be applied to the filter selected (Fig. 7). Since we are defining the action for the "Allow Terminal Services" filter, select the "Permit" radio button and click the "Next" button.
  2. Click the "Finish" button to complete the linking of the "Permit Filter Action" to the "Allow Terminal Services" filter.

Fig 7: IP Filter Actions List

You next need to link the "DENY" filter action to the "Deny Terminal Services" filter. Follow the same steps above and choose the "Deny Terminal Services" radio button at the "IP Filter List" screen and the "DENY" radio button at the Filter Action screen. Figure 8 displays the completed IP Security Rules list.

Fig 8: Completed IP Security Rules List

The checkmarks besides the filters indicate that this rule is active. You see that the <Dynamic> filter rule is not checked. This is the default response rule that was unchecked during the initial set-up of this new policy.

At this point, we are ready to activate this new IP Security policy to our machine. Click the "Close" button and return to the Group Policy editor screen. You should now see the new policy that you created in the listing. Right-click on this new policy and select the "Assign" menu item. If done properly, the folder icon next to your policy should have a small green plus (+) sign (figure 9) and a notation under the "Policy Assigned" column that it is applied.

Fig 9: New IP Security Policy Applied

Testing and Expanding this New Policy

This new policy, which allows Terminal Services from the defined subnet only, is now active. All other services to and from this host are unaffected by this policy since it was applied as a default allow scheme in which all traffic is allowed by default, with restrictions to certain services. A little more configuration (and some noted caveats) will be required to change this into a default deny scheme, with only defined services and defined hosts/subnets allowed to communicate to this host. To change to a default deny scheme, some planning must first be performed to ensure that you don't accidentally deny a service on the local host.

  • Identify all protocols and ports used by the listening services on your host that you wish to permit access to;
  • Define the hosts/subnets you wish to connect to this service; and,
  • Work within the caveats of the IP Security Filters.

After you create the filters for your services, create one more filter with the following parameters to deny all traffic that is not previously allowed.

  • Source: Any
  • Destination: My IP Address
  • Protocol: Any
  • Filter Action: DENY

Since this filter is the least descriptive of any filter in the list, it will only be applied if the traffic does not match any other defined filter.

So, What Are the Caveats?

Since the IP Security Filters are simply packet filters, they do not keep track of the connections and do not automatically allow the reverse connections. For example, if you define a filter that allows HTTP traffic from your local subnet(s) to your host, but you also wish to allow this machine to browse Web sites outside of your local subnet, care must be taken in building the rules to ensure that the source and destination ports are defined correctly for each instance. This is why it is often easier to create individual filters for each instance you wish to allow for. In this case, create one filter that has a specific subnet defined as the source, "My IP Address" as the destination and TCP/80 as the destination Protocol/Port (don't forget to create the reverse of this rule also). Then create a second filter with the source as "My IP Address" as the source, "Any" as the destination and TCP/80 as the destination Protocol/Port.

Don't neglect your friend, ICMP. Since ICMP is widely used to test if a machine is reachable, among other notifications, take care when defining your filters to include ICMP support. Since the IP Security filters lump all ICMP type codes together, you cannot create rules that only allow ICMP echo/echo-reply requests. For example, you cannot create a filter that allows PING from the local subnet to your host, and allow PING from your local host to the Internet. By attempting to create such a rule, you would be allowing ICMP communications from both the local subnet and the Internet. One solution to this would be to define a filter that will "PERMIT" ICMP from a specific subnet and a second filter that will "BLOCK" ICMP from "Any". Such a rule would not allow hosts on the Internet to PING your host, but you would not PING them either!

The "BLOCK" filter action does not produce any Windows Event messages, so you will not be able to log the denied attempts. It is suggested that users test out the IP Security filters on non-production machines before applying to a production host to ensure that they operate in the expected fashion. To test the filters, simply try your defined connection attempts to your defined services from both the allowed subnets and the places where connections should be denied. If something is not working correctly, simply double-click on the security policy in the Group Policy editor and uncheck the box next to the rule in question. This action of checking and unchecking rules is applied immediately, so there is no need to exit and re-apply the IP Security Policy. If you wish to disable the IP Security policy altogether, simply right-click on the policy in the Group Policy editor and select the "Un-Assign" menu item. The policy will be removed, and full access to the host will be resumed without any restrictions.

Get Rid of the GUI

Setting up IP Security filters on numerous hosts can be tedious, especially if the filters will all be defined the same way. Fortunately, the Windows 2000 Resource Kit includes a command line utility called IPSECPOL.EXE that allows you to create and apply an IP Security policy and filters from the command line and batch files.

There are a few caveats when using the IPSECPOL.EXE command line utility, one of which is that it will not use your defined DENY filter action, but will add an explicit BLOCK for each filter that specifies the "-n BLOCK" flag. I will not go into all the command line options to the IPSECPOL.EXE utility, for there is an extensive on-line help system available by typing IPSECPOL.EXE -?, but I have included a sample batch file with in-line comments which can be tailored for your environment. Due to line wrapping, some cleanup will need to be performed, as well as defining your actual subnets:

REM I make no warranty or claim against errors contained within this batch file
if "%1"=="" goto noparam
echo **** This batch file relies on the IPSECPOL.EXE ****
echo **** Located in the Windows 2000 Resource Kit ****
echo This batch file will install a default IPSEC policy on Windows 2000 machines
echo This default policy allows the following:
echo - Allow Terminal Services from Subnet only
echo - Deny Terminal Services from everywhere else
echo - Allow all IP traffic from local subnet
echo - Allow Netbios to entire Class A subnet
echo - Allow HTTP/HTTPS to entire Class A subnet and the Internet
echo - Allow ICMP entire Class A subnet
echo - Deny ICMP to/from Internet
echo - Deny all other traffic not defined above
echo If you need special access to this server from other subnets
echo other than the user-defined networks, or for other serivces not
echo listed above, please add the appropriate filters to the
echo My-Host-Filters IP Security policy
if "%2"=="/s" echo You have chosen to allow NetBios connections to be established from
the entire CLass A subnet
if "%2"=="/s" echo to this server from the entire user-defined network
echo This is your last chance to cancel this operation.
echo Press Ctrl-C to exit this batch file, or
REM Allow Terminal Services from the local network
ipsecpol -w REG -p "My-Host-Filters" -r "Allow Terminal Services from the local subnet" -
f -f %1:3389+ -n PASS
REM Deny Terminal Services from anywhere else
ipsecpol -w REG -p "My-Host-Filters" -r "Deny Terminal Services from ANY" -f *+%1:3389 -f
%1:3389+* -n BLOCK
REM Allow all IP from network
ipsecpol -w REG -p "My-Host-Filters" -r "Allow all IP from network" -f -n PASS
REM Allow Netbios/CIF to/from entire Class A network (build forward and reverse
rules) if /s flag is used on command-line
if "%2"=="/s" ipsecpol -w REG -p "My-Host-Filters" -r "Allow NetBios to/from entire Class A network" -f -f -f -f -f -f
%1:135+*:TCP -f %1:135+*:UDP -f
%1:137+*:UDP -f %1:139+*:TCP -f
%1:445+*:TCP -f*:TCP -f*:UDP -f*:UDP -f*:TCP -f*:TCP -f
%1+ -f %1+ -f
%1+ -f %1+ -f
%1+ -n PASS
REM If /s is not used on command line, only allow Netbios/CIF to the Class A
network, but don't let NetBios in to my host
if not "%2"=="/s" ipsecpol -w REG -p "My-Host-Filters" -r "Allow NetBios to entire
network only" -f %1+ -f %1+ -f
%1+ -f %1+ -f
%1+ -f*:TCP -f*:UDP -f*:UDP -f*:TCP -f*:TCP -n PASS
REM Allow HTTP/HTTPS to entire Class A network
ipsecpol -w REG -p "My-Host-Filters" -r "Allow HTTP/HTTPS to entire Class A
network" -f %1+ -f %1+ -n PASS
REM Allow DNS Lookups to entire CLass A network
ipsecpol -w REG -p "My-Host-Filters" -r "Allow DNS Lookups" -f
%1+ -f %1+ -f*:TCP -f*:UDP -n PASS
REM Allow HTTP/HTTPS to Internet
ipsecpol -w REG -p "My-Host-Filters" -r "Allow HTTP/HTTPS to Internet" -f %1+*:80:TCP -f
%1+*:443:TCP -n PASS
REM Allow ICMP to/from entire Class A network
ipsecpol -w REG -p "My-Host-Filters" -r "Allow ICMP to/from entire Class A
network" -f %1+*:ICMP -n PASS
REM Block all other access from anywhere
ipsecpol -w REG -p "My-Host-Filters" -r "Deny all other traffic" -f *+%1 -n BLOCK
REM Activate Policy
ipsecpol -w REG -p "My-Host-Filters" -x
echo The My-Host-Filters IPSEC policy has been applied and activated
goto end
echo **** This batch file relies on the IPSECPOL.EXE ****
echo **** Located in the Windows 2000 Resource Kit ****
echo IPSEC.BAT v1.0
echo USAGE: ipsec.bat YOUR HOST IP ADDRESS [/s]
echo To enable the default IPSEC filters:
echo ipsec.bat
echo To enable the default IPSEC filters AND to serve NetBios to the Class A
echo ipsec.bat /s
goto end


This concludes our look at IP security policies. The upcoming second installment of this series will continue our two-part overview of implementing Windows 2000 IP Security filters by discussing encryption of Windows systems and implementing IP security filters.

Joe Klemencic is currently performing Data Security responsibilities at Fermi National Accelerator Laboratory in Batavia, Illinois. He has spent the past 11 years in network architecture support, design and security.

This article originally appeared on SecurityFocus.com -- reproduction in whole or in part is not allowed without expressed written consent.

0 Favorited
0 Files

Tags and Keywords

Related Entries and Links

No Related Resource entered.