Load Balancing and Fault Tolerance for Transport Servers
Microsoft Exchange Server 2007 will reach end of support on April 11, 2017. To stay supported, you will need to upgrade. For more information, see Resources to help you upgrade your Office 2007 servers and clients.
Applies to: Exchange Server 2007, Exchange Server 2007 SP1, Exchange Server 2007 SP2, Exchange Server 2007 SP3
This topic describes load balancing mechanisms and fault tolerance options for message routing with Microsoft Exchange Server 2007 transport servers. In Exchange 2007, load balancing and fault tolerance options with message routing occur automatically to increase the availability of transport servers for efficient mail flow and delivery in the Exchange organization.
Exchange 2007 routing uses deterministic algorithms to select the least cost routing path over which to route messages to remote Active Directory sites, Send connectors, and remote routing groups. For more information about how the least cost routing path is calculated, see Understanding Active Directory Site-Based Routing.
After the least cost routing path to a destination is chosen, load balancing and fault tolerance mechanisms are useful in several different message routing scenarios. All the message routing scenarios in which Exchange 2007 provides load balancing and fault tolerance follow a common approach. If more than one transport server is available, round robin load balancing is used. For example, when more than one Hub Transport server exists in a remote Active Directory site, round robin load balancing determines the routing path. Fault tolerance is achieved by connecting to the next available server in a prioritized list of servers when the selected server is unavailable.
Note
If more than one routing path results in the same aggregate cost, Exchange 2007 does not load balance along these paths. Exchange 2007 always chooses one routing path and routes all messages along that routing path. This consistent deterministic routing makes troubleshooting mail flow problems easier.
Message Routing Scenarios that Support Load Balancing and Fault Tolerance
This section describes the following message routing scenarios in which Exchange 2007 routing provides load balancing and fault tolerance:
Message relay where multiple source transport servers are specified on a Send connector in the same Active Directory site
Message relay from a Hub Transport server to an Edge Transport server
Message relay from an Edge Transport server to a Hub Transport server
Message relay to a remote Active Directory site
Message relay from a Mailbox server to a Hub Transport server
Message relay from a Hub Transport server across a Microsoft Exchange Server 2003 routing group connector
Message relay to third-party Simple Mail Transfer Protocol (SMTP) servers
Note
Exchange 2007 never load balances across different routing paths, where a routing path consists of IP site links, connectors, and routing group connectors. However, Exchange 2007 does load balance across different source servers or target servers of connectors and routing group connectors in most cases, although some exceptions exist. For example, Exchange 2007 does not load balance when the source servers for a Send connector are located on different Active Directory sites.
Message Relay Where Multiple Source Transport Servers are Specified on a Send Connector in the Same Active Directory Site
The load balancing mechanism that is described in this section applies to all kinds of connectors that are configured for outgoing mail on both Edge Transport servers and Hub Transport servers, such as SMTP connectors, foreign connectors, and routing group connectors.
When you specify more than one source transport server on a connector, load balancing is achieved in a round robin manner by distributing connections across the source servers. Fault tolerance is achieved by failing over to the next alternative source server when the previously attempted source server is unavailable for that connector.
In the following figure, Send connector C1 is configured to use Hub Transport server A and Hub Transport server B as source servers. When Hub Transport server C routes messages to Send connector C1, the message distribution is load balanced between Hub Transport server A and Hub Transport server B.
Multiple source transport servers on a Send connector in the same Active Directory site
.gif "Many Source Transport Servers on Send Connector")
Load balancing does not occur if the server that is relaying mail is also configured as a source transport server for the selected connector. In such instances, local server proximity takes precedence over local Active Directory site proximity, and mail is always routed by using the local server. In this figure, if Hub Transport server C is also configured as a source transport server on the Send connector C1, mail relayed from Hub Transport server C is routed over Send connector C1 instead of being load balanced to Hub Transport server A and Hub Transport server B.
Message Relay from a Hub Transport Server to an Edge Transport Server
When more than one Edge Transport server is subscribed to a single Active Directory site, all the Edge Transport servers are added as source servers to a single inbound Send connector on the Edge Transport servers. Load balancing between the Edge Transport servers is achieved much like load balancing is handled between multiple Hub Transport servers on the same Send connector.
Messages that are destined to the Internet are routed first to the Active Directory site to which the Edge Transport servers are subscribed. The receiving Hub Transport server in that site then relays the messages to one of the Edge Transport servers that are listed as source transport servers on the Send connector configured to use DNS address resolution. Connection requests are load balanced across the subscribed Edge Transport servers. If the selected server is unavailable, the connection is retried to the next Edge Transport server that is hosting the Send connector that is configured to use DNS address resolution.
Note
Inter-site relay always occurs between Hub Transport servers. Hub Transport servers in remote Active Directory sites do not directly relay to the Edge Transport server that is subscribed to another Active Directory site.
Manual Failover of an Edge Transport Server
We recommend that you subscribe more than one Edge Transport server to an Active Directory site to provide automatic fault tolerance and failover if one of the Edge Transport servers goes offline. If you can subscribe only one Edge Transport server to an Active Directory site, when the Edge Transport server goes offline, you must manually intervene to reroute Internet-bound mail through another Active Directory site.
As shown in the following figure, if Edge Transport server 1 goes offline, you can manually disable the * Connector that is configured on Edge Transport server 1 in the Active Directory directory service for Site 1. E-mail that is queued in Site 1 to Edge Transport server 1 is automatically resubmitted, categorized, and then rerouted, by using the connector selection algorithm, through one of the other Active Directory sites where an Edge Transport server is subscribed.
In this figure the mail is rerouted to Active Directory Site 2 to be routed through Edge Transport server 2. When Edge Transport server 1 becomes available again, you must re-enable its * Connector in Active Directory Site 1 so that the Internet-bound e-mail in Site 1 can be routed through Edge Transport server 1.
Manual failover of an Edge Transport server
.gif "Manual Failover of Edge")
Message Relay from an Edge Transport Server to a Hub Transport Server
When an Edge Transport server is subscribed to an Active Directory site, a Send connector is automatically created and configured on the Edge Transport server. This Send connector sends messages to the Hub Transport servers in the Active Directory site where the Edge Transport server is subscribed. This Send connector is configured to use a -- placeholder in the address space. The -- placeholder in the address space for the inbound Send connector represents the authoritative and internal relay accepted domains for the Exchange organization. The Hub Transport servers that are deployed in the Active Directory site at the time that the Edge Subscription is created are listed as smart hosts for the connector. Load balancing and fault tolerance are achieved across the Hub Transport servers that are on the list of smart hosts for the inbound Send connector.
Note
If additional Hub Transport servers are deployed in the Active Directory site after the Edge Subscription is created, these Hub Transport servers do not participate in the EdgeSync synchronization process. However, the new Hub Transport servers are added to the list of smart hosts for the inbound Send connector. For more information, see EdgeSync and Send Connectors.
Message Relay to a Remote Active Directory Site
When more than one Hub Transport server is deployed in a single Active Directory site, connections to those Hub Transport servers from other Active Directory sites are prioritized in a round robin manner. When a Hub Transport server in one Active Directory site resolves the location of a recipient to a Mailbox server in another Active Directory site, a prioritized list of the Hub Transport servers in the remote site is returned. If a Hub Transport server in an Active Directory site is unavailable, connection tries are made to the other Hub Transport servers on the prioritized list. This provides fault tolerance in an Active Directory site.
For example, when Hub Transport server A in Active Directory Site A relays a message to a Mailbox server in Active Directory Site B, Hub Transport server A receives a prioritized list of Hub Transport servers, such as Hub Transport server 1, Hub Transport server 2, and Hub Transport server 3, from Active Directory Site B. If Hub Transport server A can't connect to Hub Transport server 1, Hub Transport server A tries to connect to Hub Transport server 2. If it can't connect to Hub Transport server 2, it tries to connect to Hub Transport server 3, and so on.
If Hub Transport server B in Active Directory Site A must also relay messages to Active Directory Site B, the prioritized list of Hub Transport servers is adjusted to account for the servers that are located in Active Directory Site B. For example, the prioritized list of Hub Transport servers for Hub Transport server B may be ordered as Hub Transport server 2, Hub Transport server 3, and Hub Transport server 1 in remote Active Directory Site B. Such adjustments are made to balance the load among all the Hub Transport servers in the site whenever additional connections are established.
Message Relay from Mailbox Server to Hub Transport Server
In this scenario, more than one Hub Transport server is deployed in an Active Directory site. If a Hub Transport server is co-located with the Mailbox server, that Hub Transport server always takes precedence over other Hub Transport servers in the same site. This means that the Microsoft Exchange Mail Submission Service always notifies the local Hub Transport server. If no Hub Transport server is co-located with the Mailbox server, or if the Hub Transport server on the local Mailbox server is not available, the other Hub Transport servers in the same Active Directory site are used in a round robin manner.
Message Relay from a Hub Transport Server Across an Exchange 2003 Routing Group Connector
If a routing group connector is configured to use more than one target Exchange Transport server, Exchange 2007 routing uses the load balancing and fault tolerance mechanism that is described in the "Message Relay where Multiple Source Transport Servers are Specified on a Send Connector in the Same Active Directory Site" section earlier in this topic.
Message Relay to Third-Party SMTP Servers
If an SMTP Send connector is configured to use more than one smart host, connection requests are load balanced across the smart hosts. If a smart host is not available, fault tolerance is provided by retrying the connection to another smart host that is configured on the connector.
Scenarios Where Load Balancing and Fault Tolerance Do Not Occur
This section describes the following message routing scenarios in which Exchange 2007 transport servers do not provide load balancing and fault tolerance support:
Source transport servers in different Active Directory sites
Multiple connectors with equal cost
Distribution group expansion servers
Redundant least cost routing paths or hub sites
Source Transport Servers in Different Active Directory Sites
If the source transport servers of the Send connector that is being used to route e-mail messages are in different remote Active Directory sites, mail is not load balanced across these Active Directory sites. Instead, one Active Directory site is chosen, and mail is relayed to that site. The Active Directory site that has the lowest cost is preferred. If all the Active Directory sites have the same cost, the Active Directory site of the source transport server that is listed first in the source transport server list is chosen.
The following figure shows the message routing behavior when source transport servers from more than one Active Directory site are configured for a Send connector. In this figure, a message is routed from Active Directory Site 3 to an external recipient. Connector C1 is selected as the connector with the closest matching address space. The source transport servers for Connector C1 are the Hub Transport servers in Active Directory Site 1 and Active Directory Site 2. If the first source transport server listed is in Active Directory Site 1, all messages from Active Directory Site 3 are routed to Active Directory Site 1. Any Hub Transport server in Active Directory Site 1 can receive the message and then use local Active Directory site load balancing to distribute the messages for relay between Hub Transport server A and Hub Transport server B.
Source transport servers from different Active Directory sites configured on a Send connector
.gif "Source Transport Servers in Different AD Sites")
Load balancing is not supported across Active Directory sites because Exchange 2007 always uses deterministic routing and always selects only one Active Directory site to route messages.
Multiple Connectors with Equal Cost
If more than one equal cost connector is available to route messages, messages are not load balanced between those connectors. Exchange 2007 routing deterministically chooses a connector by using the selection algorithms that are described in Understanding Active Directory Site-Based Routing.
Distribution Group Expansion Servers
You can configure a distribution group to use a specific expansion server. If you specify an expansion server, all messages to the distribution group are routed to the specified expansion server. The expansion server expands the group membership, resolves each recipient, and routes the messages. Load balancing across more than one expansion server is not supported. If the expansion server is unavailable, the messages are queued at the point of failure and the queue is put into a retry state.
Redundant Least Cost Routing Paths or Hub Sites
After Exchange 2007 routing has calculated the least cost routing path and has made a routing path selection based on the criteria that is described in Understanding Active Directory Site-Based Routing, Exchange 2007 routing does not recalculate the routing path unless the configuration data changes. If a connection cannot be made by using this deterministic routing path, Exchange 2007 routing does not try to calculate an alternative routing path. In this case, the messages are queued at the point of failure and rerouted.
The following figure shows how message routing occurs in this scenario in an Active Directory site topology.
A message that is sent from Active Directory Site 1 to Active Directory Site 4 has two available paths, each of which results in the same cost. However, the path Site 1-Site 2-Site 4 is chosen because Active Directory Site 2 is alphanumerically lower than Active Directory Site 3.
Redundant least cost routing paths or Hub sites
.gif "Redundant Least Cost Paths or Hub Sites")
In this topology, Active Directory Site 2 is also configured as a Hub Transport server site. This configuration forces message delivery to be relayed through that site because it exists along the selected least cost routing path. If messages that are being sent from Site 1 to Site 4 cannot be relayed from Site 1 to Site 2 for any reason, such as network connectivity failure between Site 1 and Site 2, all messages are queued at Site 1.
If Site 2 is not a Hub Transport server site, mail is delivered directly from Site 1 to Site 4. Direct relay is not affected by the lack of network connectivity between Site 1 and Site 2. Direct relay works as long as there is a network layer route from Site 1 to Site 4. The network layer of the Exchange topology between the sites defines the path that the computers use to send data to one another. However, in this figure, because Site 2 has a Hub Transport server in its site, all messages from Site 1 to Site 4 must relay through Site 2. In this scenario, Exchange 2007 does not support switching to an alternative equal cost routing path but relies on the IP network layer redundancy and fault tolerance between the sites for message relay. The network layer is expected to be resilient against physical link failures and provide redundant alternative paths to a destination.
SMTP Connection Management
This section explains SMTP connection management in the context of load balancing and fault tolerance for Exchange 2007. The Hub Transport server makes connection requests to remote servers by using SMTP. The remote server may be a Hub Transport server in a different Active Directory site, a smart host, or an Edge Transport server.
For example, if 60 messages are queued for relay to a remote Active Directory site and that site has three Hub Transport servers, the Exchange transport component making the connection balances the message relay load among all those servers. One connection is established to each server, and each connection is used to transfer about approximately 20 messages. The transfer rate depends on the network bandwidth and the size of the messages.
The number of messages transferred by each connection is not configurable. However, the maximum number of connections per queue can be constrained by two configuration settings on the transport server: MaxPerDomainOutboundConnections and MaxOutboundConnections. MaxPerDomainOutboundConnections limits the number of connections that can be established per queue. MaxOutboundConnections limits the total number of outbound connections that can be established by the server. You can configure these settings by using the Set-TransportServer cmdlet in the Exchange Management Shell and the transport server property pages in the Exchange Management Console.
For more information, see the following topics:
For More Information
For more information, see the following resources: