Authentication methods

Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2

Authentication methods

Each rule defines a list of authentication methods. Each authentication method defines the requirements for how identities are verified in communications to which the associated rule applies. The two peers must have at least one common authentication method or communication will fail. Creating multiple authentication methods increases the chance that a common method between two computers can be found.

Only one authentication method can be used between a pair of computers, regardless of how many are configured. If you have multiple rules that apply to the same pair of computers, you must configure the authentication methods list in those rules to enable the pair to use the same method. For example, if a rule between a pair of computers specifies only Kerberos for authentication and filters only TCP data and, in another rule, specifies only certificates for authentication and filters only UDP data, authentication will fail.

Overview of authentication methods

For authentication, IPSec allows you to use the Kerberos V5 protocol, certificate-based authentication, or preshared key authentication.

  • The Kerberos V5 security protocol is the default authentication technology. This method can be used for any clients running the Kerberos V5 protocol (whether or not the clients are running Windows 2000, Windows XP Professional, or a Windows Server 2003 operating system) that are members of the same or trusted domains.

  • A public key certificate should be used in situations that include Internet access, remote access to corporate resources, external business partner communications, or computers that do not run the Kerberos V5 security protocol. This requires that at least one trusted certification authority (CA) and associated certificate have been configured. Computers running Windows 2000, Windows XP, or a Windows Server 2003 operating system support X.509 Version 3 certificates, including computer certificates generated by commercial CAs.

  • A preshared secret key can be specified. It is simple to use and does not require the client to run the Kerberos V5 protocol or have a public key certificate. Both parties must manually configure IPSec to use this preshared key.


    • The use of preshared key authentication is not recommended because it is a relatively weak authentication method. Preshared key authentication creates a master key that is less secure (that might produce a weaker form of encryption) than certificates or the Kerberos V5 protocol. In addition, preshared keys are stored in plaintext. Preshared key authentication is provided for interoperability purposes and to adhere to IPSec standards. It is recommended that you use preshared keys only for testing and that you use certificates or Kerberos V5 instead in a production environment.


  • The ability to authenticate using Kerberos V5 is not supported on computers running Windows XP Home Edition or computers running any other version of Windows 2000, Windows XP, or a Windows Server 2003 operating system that are not a member of an Active Directory domain.

  • For information about how to configure authentication methods, see Define IPSec authentication methods.

IPSec certificate to account mapping

When you use either Kerberos V5 or certificate authentication, you can set restrictions on which computers are allowed to connect. When you enable IPSec certificate to account mapping, the IKE protocol associates (maps) a computer certificate to a computer account in an Active Directory domain or forest, and then retrieves an access token, which includes the list of user rights assigned to the computer. You can restrict access by configuring Group Policy security settings and assigning either the Access this computer from the network user right or the Deny access to this computer from the network user right to individual or multiple computers as needed.

When a Windows 2000 or a Windows Server 2003 family certification authority is used for computer certificate autoenrollment, the certificates are automatically stored in Active Directory as a property of the computer account. Alternatively, you can use a non-Microsoft certification authority in which to enroll computers. If you do not use computer certificate autoenrollment, you must manage certificate to account mapping manually.


  • If you use the Kerberos V5 protocol for authentication, the IKE protocol verifies access controls, but only if IKE is a responder. Therefore, for access controls to be applied, you must configure the IPSec policy on the client computer to initiate IKE to the server. If you use certificate authentication, access controls are applied if either the server or the client computer initiates IKE to the server.

  • For information about configuring security settings in Group Policy, see Security settings overview.