Disabling TLS Between Active Directory Sites to Support WAN Optimization

In Microsoft Exchange Server 2007, Transport Layer Security (TLS) encryption is mandatory for all SMTP communication between Hub Transport servers. This increases overall security of hub-to-hub communications. However, in certain topologies where WAN Optimization Controller (WOC) devices are used, the TLS encryption of SMTP traffic may be undesirable. Exchange Server 2010 supports disabling TLS for hub-to-hub communications for these specific scenarios.

Consider the topology shown in the following figure. The assumption for this four-site topology is that the two central office sites and Branch Office 2 are well-connected, whereas the connection between Central Office Site 1 and Branch Office 1 is over a WAN link. The company has installed WOC devices on this link to compress and optimize traffic over the WAN.

Sample network topology with WOC devices

In this topology, because Exchange 2010 uses TLS encryption for communication between Hub Transport servers, the SMTP traffic that goes over the WAN link can't be compressed. Ideally, all SMTP traffic that goes over the WAN link should be unencrypted SMTP, while retaining TLS security within well-connected sites. Exchange 2010 gives you the option to disable TLS encryption for traffic between sites by configuring Receive connectors. Using this ability in Exchange 2010, you can configure an exception to the SMTP traffic between Central Office Site 1 and Branch Office 1, as shown in the following figure.

Preferred logical message flow

The recommended configuration is to limit the non-encrypted SMTP traffic to only those messages that pass over the WAN link. Therefore, the intrasite hub-to-hub traffic in all sites, and the cross-site hub-to-hub communications that don't involve Branch Office 1 should all be TLS encrypted.

To achieve this end result, you need to do the following, in the specified order, on every Hub Transport server in the sites that contain the WOC devices (Central Office Site 1 and Branch Office 1 in the sample topology):

1. Enable downgraded Exchange Server authentication.
2. Create a Receive connector to handle the traffic over the connection that has WOC devices.
3. Configure the remote IP address range property of the new Receive connector to the IP address ranges of the Hub Transport servers in the remote site.
4. Disable TLS on the new Receive connector.

In addition, you need to do the following to ensure all SMTP traffic over the WAN is handled by the new Receive connectors you created:

• Configure the sites that will participate in the non-TLS communication as hub sites to force all message flow through the new Receive connectors (Central Office Site 1 and Branch Office Site 1 in the sample topology).
• Verify that the IP site link costs are configured in a way that ensures the least cost routing path to your remote site (Branch Office 1 in the sample topology) goes through the network link that has the WOC devices.

The following sections provide an overview of each of these steps. For step-by-step instructions on how to configure your Hub Transport servers to disable TLS, see Suppress Anonymous TLS Connections.

Looking for management tasks related to managing transport servers? See Managing Transport Servers.

Contents

Downgrading Authentication over TLS-Disabled Connections

Creating and Configuring Receive Connectors

Creating Hub Sites

Configuring Site Link Costs

Downgrading Authentication over TLS-Disabled Connections

Kerberos authentication is used with TLS encryption in Exchange. When you disable TLS on hub-to-hub communications, you need to perform another form of authentication. When Exchange 2010 communicates with other servers running Exchange that don't support X-ANONYMOUSTLS, such as the Exchange Server 2003 servers, it falls back to using GSSAPI (Generic Security Services Application Programming Interface) authentication. All communications between Exchange 2010 Hub Transport servers use X-ANONYMOUSTLS. By configuring your Hub Transport server to use downgraded Exchange Server authentication, you are in effect enabling it to use GSSAPI when communicating with other Exchange 2010 Hub Transport servers.

Return to top

Creating and Configuring Receive Connectors

You need to create Receive connectors that will solely be responsible for handling the non-TLS encrypted traffic. Using separate Receive connectors for this purpose ensures that all traffic that doesn't pass through the WAN link remains protected by TLS encryption.

To restrict the new Receive connectors to only the traffic over the WAN, you need to configure the remote IP address range property. Exchange always uses the connector with the most specific remote IP address range. Therefore, these new connectors will be preferred over the default Receive connector configured to receive messages from anywhere.

Going back to the sample topology, assume that the class C subnet 10.0.1.0/24 is used for the Central Office Site 1 and 10.0.2.0/24 is used for the Branch Office 1. To prepare for disabling TLS between these two sites, you need to:

1. Create a Receive connector (called WAN) on each Hub Transport server in Central Office Site 1 and Branch Office 1.
2. Configure the remote IP address range of 10.0.2.0/24 on each new Receive connector in Central Office Site 1.
3. Configure the remote IP address range of 10.0.1.0/24 on each new Receive connector in Branch Office 1.
4. Disable TLS on all of the new Receive connectors.

The end result is shown in the following figure (with the remote IP address range property of the WAN Receive connectors shown in parentheses). Only a single Hub Transport server is shown in Branch Office 1, and Branch Office 2 is omitted for clarity purposes.

Receive connector configuration

Return to top

Creating Hub Sites

By default, an Exchange 2010 Hub Transport server will attempt a direct connection to the Hub Transport server closest to the final destination of a specific message. In the sample topology, if a user in Branch Office 2 sends a message to a user in Branch Office 1, the Hub Transport server in Branch Office 2 will connect to the Hub Transport server in Branch Office 1 directly to deliver that message. That connection will be encrypted and therefore not desirable in the specific topology. To have such messages pass through the Hub Transport servers on Central Office Site 1, thereby ensuring they aren't encrypted while in transit over the WAN link, Central Office Site 1 and Branch Office 1 need to be configured as hub sites. In short, any site where you have a Hub Transport server with a Receive connector with TLS disabled needs to be configured as a hub site, so you can force servers in other sites to route traffic through that site. For more information about hub sites, see "Implementing Hub Sites" in Understanding Message Routing.

Return to top

Configuring Site Link Costs

Configuring hub sites alone isn't sufficient to ensure that all traffic is unencrypted over the WAN link. This is because Exchange will route messages through hub sites only if the hub site lies within the least cost routing path. For example, assume that the IP site link costs for our sample topology are configured in Active Directory as shown in the following figure (Central Office Site 2 is omitted for clarity).

IP site link costs for the sample topology

In this case, the path from Branch Office 2 to Branch Office 1 that goes through the hub site has a total cost of 12 (6+6), whereas the cost of the direct path is 10. Therefore, none of the messages from Branch Office 2 to Branch Office 1 will go through Central Office Site 1 and therefore all of that traffic will still be TLS encrypted.

To avoid this issue, you need to designate an Exchange-specific cost that is higher than 12 for the IP site link between Branch Office 2 and Branch Office 1, as shown in the following figure. This will ensure that all messages go through the unencrypted channel between Central Office Site 1 and Branch Office 1.

Sample topology configured with Exchange-specific IP site link costs

For more information about configuring Exchange-specific IP site link costs, see "Controlling IP Site Link Costs" in Understanding Message Routing.

Return to top