• Article

The Cable GuyThe Authenticated Internet Protocol

Joseph Davies

Portions of this column discuss a prerelease version of Windows Server 2008. Those details are subject to change.

Windows Vista and Windows Server 2008 both support Authenticated Internet Protocol (AuthIP), an enhanced version of the Internet Key Exchange (IKE) protocol. Both AuthIP and IKE are protocols used to determine keying material and negotiate security parameters for communications protected using

Internet Protocol security (IPsec). AuthIP provides simplified IPsec policy configuration and maintenance in many configurations and offers additional flexibility for IPsec peer authentication. This article describes the protocol details of AuthIP and coexistence behaviors between IPsec peers that support either both AuthIP and IKE or only IKE.

IKE Overview

Negotiation Examples

To demonstrate the coexistence behavior, let's examine what happens between Windows Vista and Windows XP-based IPsec peers when attempting to negotiate IPsec protection.

Example 1: Two Windows Vista-based IPsec peers in request mode

In this example, a Windows Vista-based IPsec peer (Peer 1) is the initiator, and the responder is running Windows Vista (Peer 2). Both peers are configured with request mode for both inbound and outbound communications. In request mode, an IPsec peer requests IPsec protection but does not require it. The messages exchanged are the following:

  1. Peer 1 sends a clear text TCP synchronize (SYN) segment, initiating a TCP connection with Peer 2.
  2. Peer 2 sends a TCP SYN-Acknowledgment (ACK) segment.
  3. Peer 1 sends a TCP ACK segment.
  4. Peer 1 sends an AuthIP-based ISAKMP message, initiating AuthIP main-mode negotiation.
  5. Peer 1 sends an IKE-based ISAKMP message, initiating IKE main-mode negotiation.
  6. Peer 2 responds with an AuthIP-based ISAKMP message, continuing AuthIP main-mode negotiation.
  7. Peers 1 and 2 complete AuthIP main-mode, quick-mode, and extended-mode (optional) negotiation.
  8. Subsequent segments sent over the TCP connection are protected with IPsec.

The exact order of Messages 1 through 5 depends on network latency. The examples described in this article are for a very low latency network, in which the TCP handshake (Messages 1 through 3) completes before Peer 1 can send the initial ISAKMP messages (Messages 4 and 5). On a higher-latency network, you would see Peer 1 send the clear text TCP SYN segment, the AuthIP-based ISAKMP message, and then the IKE-based ISAKMP message as the first three messages of the message exchange.

Example 2: Windows Vista-based IPsec peer in request mode and a Windows XP-based IPsec peer in request mode

In this example, a Windows Vista-based IPsec peer (Peer 1) is the initiator and the responder is running Windows XP (Peer 2). Both peers are configured with request mode for both inbound and outbound communications. The messages exchanged are the following:

  1. Peer 1 sends a clear text TCP SYN segment, initiating a TCP connection with Peer 2.
  2. Peer 2 sends a TCP SYN-ACK segment.
  3. Peer 1 sends a TCP ACK segment.
  4. Peer 1 sends an AuthIP-based ISAKMP message, initiating AuthIP main-mode negotiation.
  5. Peer 1 sends an IKE-based ISAKMP message, initiating IKE main-mode negotiation.
  6. Peer 2 responds with an IKE-based ISAKMP message, continuing IKE main-mode negotiation.
  7. Peers 1 and 2 complete IKE main-mode and quick-mode negotiation.
  8. Subsequent segments sent over the TCP connection are protected with IPsec.

Example 3: Windows Vista-based IPsec peer in request mode and a Windows XP-based IPsec peer in require mode

In this example, a Windows Vista-based IPsec peer (Peer 1) is the initiator and the responder is running Windows XP (Peer 2). Peer 1 is configured with request mode for outbound communications and Peer 2 is configured with require mode for inbound communications. In require mode, an IPsec peer requires IPsec protection and silently discards unprotected packets. The messages exchanged are the following:

  1. Peer 1 sends a clear-text TCP SYN segment initiating a TCP connection, which Peer 2 silently discards.
  2. Peer 1 sends an AuthIP-based ISAKMP message initiating AuthIP main-mode negotiation, which Peer 2 silently discards.
  3. Peer 1 sends an IKE-based ISAKMP message, initiating IKE main-mode negotiation.
  4. Peer 2 responds with an IKE-based ISAKMP message, continuing IKE main-mode negotiation.
  5. Peer 1 and Peer 2 complete IKE main-mode and quick-mode negotiation.
  6. Peer 1 retransmits the TCP SYN segment (protected with IPsec).
  7. Peer 2 sends the TCP SYN-ACK segment (protected with IPsec).
  8. Peer 1 sends the TCP ACK segment (protected with IPsec).

IKE is an Internet standard, defined in RFC 2409 (ietf.org/rfc/rfc2409.txt), that defines a mechanism to establish IPsec security associations (SAs). An SA is a combination of a mutually agreeable policy and keys that define the security services and mechanisms that help protect communication between IPsec peers. Specifically, IKE combines the Internet Security Association and Key Management Protocol (ISAKMP) in RFC 2408 and the Oakley Key Determination Protocol (RFC 2412). IKE uses the Oakley Key Determination protocol to generate secret key material for protected communications.

IKE uses ISAKMP to negotiate SAs. ISAKMP includes facilities to identify and authenticate peers, manage SAs, and exchange key material. ISAKMP is a framework for negotiating protected communications that is independent of specific key exchange protocols, encryption and integrity algorithms, and authentication methods.

IPsec peers use IKE to perform main-mode and quick-mode negotiation. Main-mode negotiation creates the ISAKMP SA, which protects security negotiations. During main-mode negotiation, the initiator and the responder exchange a series of ISAKMP messages in order to negotiate the encryption and hashing algorithms for the ISAKMP SA, exchange key determination material, and identify and authenticate each other.

In the quick-mode negotiation phase, the two IPsec SAs (one SA for inbound data and another SA for outbound data) are created, which protects the data sent between two IPsec peers. During quick-mode negotiation, the initiator and the responder exchange a series of encrypted ISAKMP messages to negotiate the encryption and hashing algorithms for both the inbound and outbound IPsec SAs.

For more information about IKE-based main-mode and quick-mode negotiation, see "IKE Negotiation for IPsec Security Associations" at microsoft.com/technet/community/columns/cableguy/cg0602.mspx.

AuthIP Overview

AuthIP is an enhanced version of IKE that offers additional flexibility with support for user-based authentication, authentication with multiple credentials, improved authentication method negotiation, and asymmetric authentication. Like IKE, AuthIP supports main-mode and quick-mode negotiation. AuthIP also supports extended mode, a part of IPsec peer negotiation during which a second round of authentication can be performed. Extended mode, which is optional, can be used for multiple authentications. For example, with extended mode you can perform separate computer-based and user-based authentications.

Multiple authentication support is part of IPsec enforcement for the Network Access Protection (NAP) platform, in which IPsec peers authenticate with computer credentials and also use a health certificate to prove that they are compliant with system health requirements. For more information about Network Access Protection, see microsoft.com/nap. And for further information about the features of AuthIP, see "AuthIP in Windows Vista" at microsoft.com/technet/community/columns/cableguy/cg0806.mspx.

AuthIP Messages

Both IKE and AuthIP use ISAKMP as their key exchange and SA negotiation protocol. ISAKMP messages are sent via the User Datagram Protocol (UDP) and consist of an ISAKMP header and one or more ISAKMP payloads. The ISAKMP header contains information about the message, including the message type. Figure 1 shows the format of the ISAKMP header.

Figure 1 Format of the ISAKMP header

Figure 1** Format of the ISAKMP header **

In the ISAKMP header, the Initiator Cookie and the Responder Cookie are set to a nonzero random number chosen by the IPsec peers. The Next Payload field indicates the first payload in the ISAKMP message, just beyond the ISAKMP header. RFC 2408 lists defined values of the Next Payload field. Payload types 128-255 are reserved for private use. The Major Version and Minor Version fields indicate the versions of ISAKMP on the IPsec peer sending the ISAKMP message. For IKE and AuthIP, the major version is 1 and the minor version is 0.

The Exchange Type field indicates the type of ISAKMP exchange being performed. The type of exchange dictates the structure and order of ISAKMP payloads. RFC 2408 defines the values of the Exchange Type field. Exchange types 128-255 are reserved for private use.

The Flags field contains three flags defined in RFC 2408. The low-order bit (bit 0) is the Encryption bit, which indicates that the ISAKMP payloads are encrypted (when set to 1) or not encrypted (when set to 0). Encryption is done using the algorithm negotiated for the ISAKMP SA, which is identified by the combination of the Initiator Cookie and Responder Cookie fields. The next low-order bit (Bit 1) is the Commit bit, which indicates that the key exchange is synchronized (when set to 1) or not synchronized (when set to 0). The Commit bit is used to ensure that the SA completes its negotiation before encrypted data is sent. The next low-order bit (Bit 2) is the Authentication Only bit, which is used to indicate that the message either contains (when set to 1) or does not contain (when set to 0) the entire Notify payload of the informational exchange type and it has been authenticated but not encrypted.

The Message ID field contains a unique identifier for the message. The Message ID is used to prevent collisions when both IPsec peers attempt to simultaneously establish an IPsec SA. The Length field indicates the length of the entire ISAKMP message.

For IKE, the ISAKMP message contains a series of ISAKMP payloads. The first payload is indicated by the Next Payload field of the ISAKMP header. Each ISAKMP payload has its own Next Payload field to indicate the next payload. The Next Payload field in the last payload is set to 0. Figure 2 shows the format of IKE messages.

Figure 2 Format of IKE messages

Figure 2** Format of IKE messages **

AuthIP uses ISAKMP messages with the exchange types 243 (Main Mode), 244 (Quick Mode), 245 (Extended Mode), and 246 (Notify) in the Exchange Type field in the ISAKMP header. An important difference in AuthIP-based ISAKMP messages is that they only contain one ISAKMP payload: either the Crypto payload or the Notify payload. The Crypto payload contains the embedded payloads used for the main mode, quick mode, or extended mode negotiation. The Crypto payload can contain a set of plain text or encrypted payloads, depending on the Encryption bit in the Flags field of the ISAKMP header. Figure 3 shows the format of AuthIP messages containing the Crypto payload.

Figure 3 Format of AuthIP messages containing the Crypto payload

Figure 3** Format of AuthIP messages containing the Crypto payload **

AuthIP and IKE Coexistence

Windows Vista® and Windows Server® 2008 support both IKE and AuthIP. Windows® XP and Windows Server 2003, however, only support IKE. An initiating IPsec peer that supports both AuthIP and IKE and has a connection security policy that uses both AuthIP and IKE must determine whether the responding IPsec peer supports AuthIP or IKE. It must then use the most appropriate protocol for negotiating IPsec protection, preferring the use of AuthIP over IKE.

To determine the negotiation protocol of the responding IPsec peer, an initiating IPsec peer that uses both AuthIP and IKE sends the following messages at the same time:

  • Message 1—an AuthIP message initiating main-mode negotiation
  • Message 2—an IKE message initiating main-mode negotiation

If the responding node supports AuthIP, it must respond to Message 1 with an AuthIP message continuing main-mode negotiation and silently discard Message 2. A responding node that does not support AuthIP silently discards Message 1 because the Exchange Type field it contains a value it doesn't support, and responds to Message 2.

To prevent IKE-based negotiation between two IPsec peers running Windows Vista or Windows Server 2008 when Message 1 is dropped from the network or arrives after Message 2, IPsec peers running Windows Vista or Windows Server 2008 send Message 2 with an AuthIP Supported vendor ID ISAKMP payload. The AuthIP Supported vendor ID payload indicates that the IPsec peer supports AuthIP.

Therefore, if an IPsec peer running Windows Vista or Windows Server 2008 receives Message 2 with the AuthIP-supported vendor ID payload, it waits for the initiating IPsec peer to retransmit Messages 1 and 2, and it responds to Message 1.

The initiating IPsec peer keeps retransmitting both Messages 1 and 2 until it receives a response or times out. When the initiator receives a response, it determines the capability of the responder from the ISAKMP header of the response. If the Exchange Type is set to 243 (the exchange type for AuthIP-based main-mode negotiation), the responder is AuthIP-capable. If the Exchange Type is set to 2 (the exchange type for Identity Protection and IKE-based main mode negotiation), the responder is IKE-capable.

Based on the response, the initiator responds with either the next AuthIP message for AuthIP main mode negotiation or the next IKE message for IKE main mode negotiation. The IPsec peers must use the same protocol that was used to negotiate the ISAKMP SA for the lifetime of the SA.

Joseph Davies is a technical writer with Microsoft and is the author of the monthly online TechNet Cable Guy column at Microsoft.com/technet/community/columns/cableguy.

© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.