How to deploy an SSTP-based VPN server behind a SSL load balancer in Windows Server 2008

Consider the following scenario:

• You have a Secure Socket Tunneling Protocol (SSTP)-based VPN server that is running Windows Server 2008.
• The VPN server is located behind an SSL load balancer.
• The server is assigned a private IP address.
• The server has a public IP address that is registered to a DNS server.

This article describes how to deploy an SSTP-based VPN server behind an SSL load-balanced computer that is running Windows Server 2008.

SSTP is a new kind of VPN tunnel that is available in the Routing and Remote Access Server role in Windows Server 2008. SSTP enables the encapsulation of Point-to-Point Protocol (PPP) packets over HTTP. This lets you more easily establish a VPN connection through a firewall or through a Network Address Translation (NAT) device. This also lets you establish a VPN connection through an HTTP proxy device.

In large organizations, it is common to configure an SSL load balancer to close an HTTPS connection and then to open an HTTP connection to a Web server or to a Routing and Remote Access server.

Overview

The information in this article applies to the following network-configuration scenario:

• The SSL load balancer has an IP address of 1.2.3.4.
• The SSL load balancer has the DNS name of server.contoso.com.
• There are two Routing and Remote Access servers that are located in a perimeter network (also known as DMZ, demilitarized zone, and screened subnet). One of these servers has an IP address of 1.2.3.5, and the other server has an IP address of 1.2.3.6.

Configuration information

Warning Serious problems might occur if you modify the registry incorrectly by using Registry Editor or by using another method. These problems might require that you reinstall the operating system. Microsoft cannot guarantee that these problems can be solved. Modify the registry at your own risk.

To configure an SSTP-based VPN server in the scenario that is described in the "Overview" section, follow these steps:

1. Configure the SSL load balancer to close SSTP-based HTTPS connections that are directed to a specific uniform resource identifier (URI) in a pool of Routing and Remote Access servers. (These servers have IP addresses of 1.2.3.5 and 1.2.3.6, and they use port 80.) The following is an example of a URI:
   https://host_name:443/sra_{ BA195980-CD49-458b-9E23-C84EE0ADCD75})
2. Install a computer certificate on the Windows Server 2008-based computer that serves as the SSL load balancer. The computer certificate must have the EKU type set to Server Authentication or to All Purpose. This certificate must have a subject name (CN) that is the same as the host name to which the VPN clients connect. This configuration is required for successful SSL negotiation.
   
   For example, consider the following scenario:
   • If a VPN client is configured to connect to the public IP address of the NAT device (1.2.3.4), the subject name of the certificate must also be 1.2.3.4.
   • If a VPN client is configured to connect to the FQDN (server.contoso.com) that can be accessed publicly, the subject name of the certificate must also be server.contoso.com.
3. If you want to close the HTTP connection on the Routing and Remote Access servers, follow these steps:
   1. Set the following registry key to a value of 0 so that the Routing and Remote Access server can listen on a specific HTTP connection:
      HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Sstpsvc\Parameters\UseHTTPS
   2. By default, Routing and Remote Access servers listen on port 443 if the UseHTTPS option is set to 1 or on port 80 if the UseHTTPS option is set to 0. Complete steps 3c-3g if you want the Routing and Remote Access server to listen on another port. If you do not want the Routing and Remote Access server to listen on another port, go to step 3h.
   3. Start Registry Editor, and then locate the following registry subkey:
      HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SstpSvc\Parameters
   4. In the details pane, right-click ListenerPort, and then click Modify.
   5. Click Decimal, type an alternative port number such as 5000, and then click OK.
   6. Exit Registry Editor.
   7. Restart the Routing and Remote Access service.
   8. Even though HTTPS connections are closed on the SSL load balancer, the Routing and Remote Access server needs the hash value of the computer certificate that is installed on the SSL load balancer to improve security. To make sure that the Routing and Remote Access server has the hash value of the computer certificate that is installed on the SSL load balancer, configure the following REG_BINARY subkey by using the SHA256 certificate hash of the computer certificate that is installed on the SSL load balancer:
      HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Sstpsvc\Parameters\Sha256CertificateHash
      Note If you cannot locate the SHA256 certificate hash of the computer certificate that is installed on the SSL load balancer, complete steps 3i-3k to configure a temporary value for the Sha256CertificateHash registry key.
   9. On a different computer, create a VPN connection that is SSTP-based and for which the VPN destination is the host name of the SSL load balancer.
   10. Click Connect to connect the VPN connection. The connection attempt fails. Open Event Viewer on the client computer, and then locate the following event:
       Cryptographic binding failed for the client connection. The reason is the client and the server have mismatched certificate hashes configured. SHA1 Certificate Hash: ... SHA256 Certificate Hash: ...
   11. Note the value of SHA256 certificate hash in the event on the client computer, and then use this value for the Sha256CertificateHash registry key on the Routing and Remote Access server.
   12. Restart the Routing and Remote Access server.
   13. Reestablish the SSTP connection.
4. Install the Routing and Remote Access server role by using Server Manager on all the Routing and Remote Access servers.
5. Configure the Routing and Remote Access server by running through the Routing and Remote Access Configuration Wizard.
6. If Windows Firewall is enabled on the Routing and Remote Access server, manually open the appropriate port number, such as port 80, in Windows Firewall. If the Routing and Remote Access server's incoming and outgoing filters are enabled on the Routing and Remote Access server, manually open the appropriate port number on the Routing and Remote Access server's incoming and outgoing filters.
7. Set the Routing and Remote Access server to listen for HTTP-based VPN connections on a specific port number.

For more information, click the following article number to view the article in the Microsoft Knowledge Base: