Client computers that are behind a NAT device cannot access resources that are behind ISA Server 2004

Consider the following scenario:

• A client computer that is behind a network address translation (NAT) device uses an Internet Protocol security (IPsec) NAT traversal (NAT-T) connection to access resources.
• These resources are behind a computer that is running Microsoft Internet Security and Acceleration (ISA) Server 2004.
• ISA Server 2004 is configured as the perimeter firewall.

In this scenario, the client computer cannot access the resources that are behind the ISA Server 2004 computer. Additionally, the following error code is logged to the ISA Server Firewall service log:Note This problem occurs if IPsec is used to help secure inbound traffic on the ISA Server computer.

This problem occurs when there is a mismatch between the TCP checksum and the checksum that is calculated by ISA Server.

When IPsec NAT-T is used, the TCP checksum and the checksum that is calculated by the ISA Server do not match. Therefore, ISA Server blocks the connection.

Service pack information

ISA Server 2004, Enterprise Edition

To resolve this problem, obtain the latest service pack for ISA Server 2004, Enterprise Edition. For more information, click the following article number to view the article in the Microsoft Knowledge Base:
After you apply the service pack, follow the steps that are described in the "How to set up an exclusion list" section.

ISA Server 2004, Standard Edition

To resolve this problem, apply the hotfix that is described in the following Microsoft Knowledge Base article:

After you apply the hotfix, follow the steps that are described in the "How to set up an exclusion list" section.

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

How to set up an exclusion list

After you install the hotfix or the service pack, configure an exclusion list for the TCP checksum validation. This configuration prevents the resource access problem because inbound traffic is examined at the IPsec layer. To configure an exclusion list, follow these steps:

1. Start Notepad on the ISA Server computer.
2. Copy and then paste the following script into Notepad. This script adds a Vendor Parameter set for the IP address and port combinations that are to be excluded from the TCP checksum validation for ISA Server.

   ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
   '
   ' Copyright (c) Microsoft Corporation. All rights reserved.
   ' THIS CODE IS MADE AVAILABLE AS IS, WITHOUT WARRANTY OF ANY KIND. THE ENTIRE
   ' RISK OF THE USE OR THE RESULTS FROM THE USE OF THIS CODE REMAINS WITH THE
   ' USER. USE AND REDISTRIBUTION OF THIS CODE, WITH OR WITHOUT MODIFICATION, IS
   ' HEREBY PERMITTED.
   ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
   
   ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
   ' This script adds a new VendorParametersSets under the array root.
   ' Used to added new parameters that are required for hotfixes or service packs.
   ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
   
   Sub AddIpsecXsumBypass()
   
       ' Create the root object.
       Dim root  ' The FPCLib.FPC root object
       Set root = CreateObject("FPC.Root")
   
       'Declare the other objects needed.
       Dim array       ' An FPCArray object
       Dim VendorSets  ' An FPCVendorParametersSets collection
       Dim VendorSet   ' An FPCVendorParametersSet object
   
       ' Get references to the array object
       ' and the network rules collection.
       Set array = root.GetContainingArray
       Set VendorSets = array.VendorParametersSets
   
       On Error Resume Next
       Set VendorSet = VendorSets.Item( "{143F5698-103B-12D4-FF34-1F34767DEabc}" )
   
       If Err.Number <> 0 Then
           Err.Clear
   
           ' Add the item
           Set VendorSet = VendorSets.Add( "{143F5698-103B-12D4-FF34-1F34767DEabc}" )
           CheckError
           WScript.Echo "New VendorSet added... " & VendorSet.Name
   
       Else
           WScript.Echo "Existing VendorSet found... value- " & VendorSet.Value("IpAddrToBypassIPSecXsum")
       End If
   
       WScript.Echo "Initial IpAddrToBypassIPSecXsum- " & VendorSet.Value("IpAddrToBypassIPSecXsum")
   
       Err.Clear
   '
   '       The format for setting an exception is: sourceip:sourceport>destinationip:destinationport
   '       Any field (entire ip or port) which is 0 is ignored.
   '       For example- 12.0.0.0:0>14.15.16.17:80;
   '       means that traffic from 12.0.0.0, any source port, to 14.15.16.17
   '       port 80 is not subject to validation of tcp checksum. The ';' character
   '       must end each address pair, even if there is only one pair.
   ''
   '
   '	THIS FOLLOWING LINE IS THE ONE THAT YOU NEED TO CHANGE	
   '
       VendorSet.Value("IpAddrToBypassIPSecXsum") = "12.34.55.66:45587>54.127.232.0:80;"
   '
   '	Use the following value to reset the exception list.
   '
   '    VendorSet.Value("IpAddrToBypassIPSecXsum") = ""
   
       If Err.Number <> 0 Then
           CheckError
       Else
           VendorSets.Save false, true
           CheckError
   
           If Err.Number = 0 Then
               WScript.Echo "Done with IpAddrToBypassIPSecXsum, saved!"
           End If
       End If
   
      WScript.Echo "Final IpAddrToBypassIPSecXsum- " & VendorSet.Value("IpAddrToBypassIPSecXsum")
   
   End Sub
   
   Sub CheckError()
   
       If Err.Number <> 0 Then
           WScript.Echo "An error occurred: 0x" & Hex(Err.Number) & " " & Err.Description
           Err.Clear
       End If
   
   End Sub
   
   AddIpsecXsumBypass
   
   'End of Script
   

3. Amend the line in the script that starts with �VendorSet.Value("IpAddrToBypassIPSecXsum")� to include the exception list. Use the following format:
   SourceIp:SourcePort>DestinationIp:DestinationPort
   For example, assume that traffic is directed from address 12.0.0.0 and from any source port to address 14.15.16.17 and to port 80. This traffic is not validated by ISA Server for the TCP checksum when you include the following line in the script:
   "12.0.0.0:0>14.15.16.17:80�
   Notes
   • Full wildcard values are supported to exclude an IP address and a port. For example, IP address 0.0.0.0 is supported and will add every client IP address to the exclusion list.
   • Multiple entries are supported. Each entry must end by using a colon (;). For example, the following address and port notation is supported:
     "12.0.0.0:0>14.15.16.17:80;12.0.0.0:1>14.15.16.17:80;"
4. On the File menu, click Save As.
5. In the File name box, type ISA_script_name.vbs, and then click OK.
6. At a command prompt, type the following command, and then press ENTER:
   ISA_script_name.vbs
   Note You must run the script from the location where you saved the ISA_script_name.vbs file.
7. Restart the Firewall service.

For example, when you want to help secure a Web listener by using IPsec, you must use an exclusion list for the TCP checksum validation. This configuration may be required for RPC over HTTP traffic when client certificate authentication is required but is not supported by Microsoft Office Outlook. In this scenario, you can use IPsec to provide certificate authentication at the network layer.

In this scenario, the exclusion list must include every client IP port and every client source port that sends traffic to the ISA Server computer on port 443. For example, the ISA_script_name.vbs script must include the following line:Note ISAWebListenerIP is the IP address that ISA Server uses for its Web listener. ISA Server listens on port 443.

For more information, click the following article number to view the article in the Microsoft Knowledge Base: