How the default response rule for IPsec policies functions in Windows Vista and in Windows Server 2008 Beta 3

This article describes how the default response rule for Internet Protocol security (IPsec) policies functions in Windows Vista and in Windows Server 2008 Beta 3. It also describes how to create a replacement rule for the default response rule in Windows Vista and in Windows Server 2008 Beta 3.

The default response rule is applied on remote computers that request an IPsec connection when no other rules are available in an IPsec policy. To communicate over an encrypted channel, the computer must respond to requests to establish an encrypted channel. The default response rule is enabled automatically when you create an IPsec policy in versions of Windows that are earlier than Windows Vista and Windows Server 2008 Beta 3. By default, the default response rule is not enabled when you create an IPsec policy in Windows Vista or in Windows Server 2008 Beta 3.

You can use the IP Security Policy Management snap-in or the netsh ipsec command to configure the default response rule when you create or modify an IPsec policy in Windows Vista or in Windows Server 2008 Beta 3. Even though you can configure the default response rule for an IPsec policy in Windows Vista or in Windows Server 2008 Beta 3, the rule is invalid when you apply an IPsec policy on a Windows Vista-based computer or on a Windows Server 2008 Beta 3-based computer. The rule is valid only when you apply the default response rule in earlier versions of Windows. Also, you cannot use the netsh ipsec command in Windows Vista or in Windows Server 2008 Beta 3 to modify the default response rule for an IPsec policy that is applied on a Windows Vista-based computer. However, you can modify the default response rule for clients that run versions of Windows that are earlier than Windows Vista. If an IPsec policy that contains the default response rule is assigned to a Windows Vista-based computer, the policy is not applied. Also, the following event is logged in the Security log:

Event ID: 5461
Source: Security Auditing
Type: Audit Failure
Message: PAStore Engine failed to apply local registry storage IPsec policy on the computer.
Policy: Policy Name
Error Code: Error Code. The parameter is incorrect

Instead of the default policy rule, you can create a rule for a policy that enables the Windows Vista-based computer or the Windows Server 2008 Beta 3-based computer to apply the rule on Windows Vista-based clients that request IPsec communication. The default response rule responds only to client requests. However, the rule that you create can respond to all kinds of requests. This rule also establishes communication over an encrypted channel with the client. If communication over an encrypted channel cannot be established during IPsec negotiations, the communication will continue in clear text over the network. This communication will not be dropped.

To create a new rule for an IPsec policy in Windows Vista or in Windows Server 2008 Beta 3, follow these steps:

1. Use the IP Security Policy Management snap-in to modify the IPsec policy. To do this, follow these steps:
   1. Use the appropriate method:
      • In Windows Server 2008 Beta 3, click Start, click Run, type mmc in the Open box, and then click OK.
      • In Windows Vista, click Start
        , type mmc in the Start Search box, and then press ENTER.
   2. On the File menu, click Add/Remove Snap-in. The Add or Remove Snap-in dialog box opens.
   3. In the Available snap-ins list, click IP Security Policy Management, and then click Add.
   4. In the Select Computer or Domain dialog box, click Finish.
   5. Click OK to close the Add or Remove Snap-in dialog box.
   6. In the task pane, click IP Security Policies on Local Computer.
   7. In the details pane, right-click the policy for which you want to create a new rule, and then click Properties.
   8. Click Add to add a new rule to the policy.
2. Configure the tunneling properties and the network-type properties for the new rule. To do this, follow these steps:
   1. On the Welcome page of the Security Rule Wizard, click Next.
   2. On the Tunnel Endpoint page, make sure that the This rule does not specify a tunnel option is enabled, and then click Next.
   3. On the Network Type page, make sure that the All network connections option is enabled, and then click Next.
3. Create a new IP filter list, and then configure the filter-list properties. To do this, follow these steps:
   1. On the IP Filter List page, click Add. The IP Filter List dialog box opens.
   2. In the Name box, type All IP Traffic, and then click Add.
   3. On the Welcome page of the IP Filter Wizard, click Next.
   4. On the "IP Filter Description and Mirrored property" page, add a description for the IP filter, and then click Next.
   5. On the IP Traffic Source page, make sure that Any IP Address is selected in the Source address list, and then click Next.
   6. On the IP Traffic Destination page, make sure that Any IP Address is selected in the Destination address list, and then click Next.
   7. On the IP Protocol Type page, make sure that Any is selected in the Select a protocol type list, and then click Next.
   8. On the "Completing the IP Filter Wizard" page, click Finish.
   9. Click OK to save the new filter list and to close the IP Filter List dialog box.
4. Create a new filter action, and then configure the filter-action properties. To do this, follow these steps:
   1. On the IP Filter List page, click the IP filter list that you created, and then click Next.
   2. On the Filter Action page, click Add to create a new filter action.
   3. On the Welcome page of the Filter Action Wizard, click Next.
   4. On the Filter Action Name page, type Always negotiate security in the Name box, and then click Next.
   5. On the Filter Actions General Options page, make sure that the Negotiate security option is enabled, and then click Next.
   6. On the "Communicating with computers that do not support IPsec" page, click Allow unsecured communication if a secure connection cannot be established, and then click Next.
   7. On the IP Traffic Security page, select the security method that you want to use, and then click Next.
   8. On the "Completing the IP Security Filter Action Wizard" page, click Finish.
   9. On the Filter Action page, select the filter action that you created, and then click Edit. The New Filter Action Properties dialog box opens.
   10. On the Security Methods tab, click to select the Accept unsecured communication, but always respond using IPsec check box, and then click OK.
   11. On the Filter Action page, click Next.
5. On the Authentication Method page, specify the authentication method that you want to use, and then click Next.
6. On the "Completing the Security Rule Wizard" page, click Finish.
7. Click OK to close the IPsec policy properties dialog box.