Selecting a VPN Protocol

Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

Tunneling and authentication protocols, and the encryption levels applied to VPN connections, determine VPN security. L2TP/IPSec provides the highest level of security. For a VPN design, determine which VPN protocol best meets your requirements. Windows Server 2003 supports two VPN protocols: Point-to-Point Tunneling Protocol (PPTP) and Layer Two Tunneling Protocol with Internet Protocol security (L2TP/IPSec).

PPTP

PPTP uses Point-to-Point Protocol (PPP) user authentication methods and Microsoft Point-to-Point Encryption (MPPE) to encrypt IP traffic. When used with MS-CHAP v2 for password-based authentication and strong passwords, PPTP is a secure VPN technology. For stronger authentication for PPTP connections, you can implement a PKI using smart cards or certificates and Extensible Authentication Protocol — Transport Level Security (EAP-TLS).

PPTP is widely supported and easily deployed, and it works with most network address translators (NATs).

L2TP/IPSec

The more secure of the two VPN protocols, L2TP/IPSec uses PPP user authentication methods and IPSec encryption to encrypt IP traffic. This combination uses certificate-based computer identity authentication to create IPSec security associations in addition to PPP-based user authentication. L2TP/IPSec provides data integrity, data origin authentication, data confidentiality, and replay protection for each packet.

Support for L2TP/IPSec is provided with Windows Server 2003, as well as with Windows 2000 and Windows XP. To use L2TP/IPSec with the Microsoft® Windows® 98, Windows® Millennium Edition (Windows Me), or Windows NT® Workstation 4.0 operating system, download and install Microsoft L2TP/IPSec VPN Client (Mls2tp.exe). For information about Mls2tp.exe, see the Microsoft L2TP/IPSec VPN Client link on the Web Resources page at https://www.microsoft.com/windows/reskits/webresources.

Table 8.1 summarizes the advantages and constraints associated with the use of the PPTP and L2TP/IPSec protocols.

Table 8.1   Advantages and Constraints of the PPTP and L2TP/IPSec VPN Protocols

Factor PPTP Advantages and Constraints L2TP/IPSec Advantages and Constraints

Client operating systems supported

Supported on clients running Windows 2000, Windows XP, Windows Server 2003, Windows NT Workstation 4.0, Windows Me, or Windows 98.

Natively supported on clients running Windows 2000, Windows XP, or Windows Server 2003.

With Mls2tp.exe installed, supported on clients running Windows 98, Windows Me, or Windows NT Workstation 4.0.

Certificate support

For EAP-TLS authentication to issue computer certificates to the authenticating server and user certificates to all VPN clients or to issue smart cards to all users, PPTP requires a certificate infrastructure.

To issue computer certificates to the VPN server and all VPN clients, L2TP/IPSec requires a certificate infrastructure or a preshared key (PSK).

Security

Provides data confidentiality. (Captured packets cannot be interpreted without the encryption key.)

Does not provide data integrity (proof that the data was not modified in transit) or data origin authentication (proof that the data was sent by the authorized user).

To increase security, use MS-CHAP v2 as the authentication protocol with strong passwords.

Offers the highest level of security, providing data confidentiality, data integrity, data origin authentication, and replay protection.

Performance

A VPN server supports more PPTP connections than L2TP/IPSec connections.

Because IPSec encryption is processing intensive, a VPN server supports fewer L2TP connections than PPTP connections. To support additional L2TP connections, increase CPU processing power or use offload network adapters.

NAT support

PPTP-based VPN clients can be located behind a NAT if the NAT includes an editor that can translate PPTP.

If you locate L2TP/IPSec–based clients or servers behind a NAT, both client and server must support IPSec NAT traversal (NAT-T).

NAT Requirements for VPN Protocols

If you are using a NAT with your VPN remote access server solution, your security plan for remote access must include the required setup for placing VPN clients behind a NAT. The VPN protocol that you deploy affects the NAT requirements.

A network address translator (NAT) translates the IP addresses and Transmission Control Protocol / User Datagram Protocol (TCP/UDP) port numbers of packets that are forwarded between a private network and the Internet. The NAT on the private network can provide IP address configuration information to the other computers on the private network.

The NAT can act as a simplified DHCP server that allocates an IP address, a subnet mask, a default gateway, and the IP address of a DNS server. The NAT can become the DNS proxy for the computers on the private network. When the NAT receives name resolution requests from a computer on the private network, it forwards the request to a specified Internet-based DNS server and returns a response to the requesting computer on the private network.

Using a NAT with PPTP connections   If a VPN client that uses a PPTP connection is behind a NAT, the NAT must include a NAT editor that can translate PPTP traffic. The NAT editor is required, because PPTP tunneled data has a Generic Routing Encapsulation (GRE) header rather than a TCP header or a UDP header. The NAT editor uses the Call ID field in the GRE header to identify the PPTP data stream and translate IP addresses and call IDs for PPTP data packets that are forwarded between a private network and the Internet.

The NAT/Basic Firewall routing protocol component of the Routing and Remote Access service includes a NAT editor for PPTP traffic.

Using a NAT with L2TP connections   IPSec NAT Traversal (NAT-T) enables IPSec peers to communicate when behind a NAT. IPSec NAT-T provides UDP encapsulation of IPSec packets to enable Internet Key Exchange (IKE) and Encapsulating Security Payload (ESP)–protected traffic to pass through a NAT. IKE automatically detects that a NAT is present and uses User Datagram Protocol — Encapsulating Security Payload (UDP-ESP) encapsulation to enable ESP-protected IPSec traffic to pass through the NAT.

To use NAT-T, both the remote access VPN client and the remote access server must support IPSec NAT–T. IPSec NAT-T is supported by Windows Server 2003 and Microsoft L2TP/IPSec VPN Client.