Understanding Public Folder Replication
Microsoft Exchange Server 2007 will reach end of support on April 11, 2017. To stay supported, you will need to upgrade. For more information, see Resources to help you upgrade your Office 2007 servers and clients.
Applies to: Exchange Server 2007, Exchange Server 2007 SP1, Exchange Server 2007 SP2, Exchange Server 2007 SP3
Public folder replication is the process by which public folder content and the public folder hierarchy is replicated across multiple servers for efficiency and fault tolerance purposes. When multiple public folder databases that are located on separate servers support a single public folder tree, Microsoft Exchange uses public folder replication to keep the databases synchronized. Public folder content exists only in Exchange stores that are configured to have a replica of a specific folder. Content and hierarchy information are replicated separately. Each public folder database retains a copy of the hierarchy, which includes lists of the other public folder databases that retain content replicas of each folder. Content replicas exist only on the public folder databases that you specify. For more information about how to configure public folder replication, see How to Configure Public Folder Replication.
Public folder databases replicate two types of public folder information:
Hierarchy Hierarchy replication occurs when the properties of the folders and organizational information about the folders have been modified. All public folder databases have a copy of the hierarchy information. Modifying the following public folder information results in hierarchy replication:
Folder name
Replica list
Folder's position in the public folder tree (including any parent and child folders)
Permissions
Note
Hierarchy replication does not occur when you change the e-mail addresses for a mail-enabled public folder. The e-mail addresses are stored on the directory object in the Active Directory directory service. Only by changing the properties within the public store database does hierarchy replication occur.
Content Content replication occurs when messages are sent to public folders or when data is added. For example, sending e-mail messages to a mail-enabled public folder or adding an organizational form to a public folder results in content replication. To replicate content, you must configure a folder to replicate its content to a specific public folder database or list of databases. Only the databases that you specify will have copies of the content. A copy of the folder that includes content is called a content replica.
How Public Folder Replication Works
When you modify a public folder or its contents, the public folder database that contains the replica of the public folder that was changed sends a descriptive e-mail message to the other public folder databases that host a replica of the public folder. To reduce traffic on your network, Exchange includes information about multiple changes in one e-mail message. The amount of information that is included in these messages depends on the size limit you set for replication messages. If any message exceeds the specified size limit, that message is sent as a separate replication message. Exchange routes these replication messages the same way that it routes other e-mail messages. For more information about public folder routing, see Routing Messages to Public Folders.
If you make changes to the public folder hierarchy that affect several folders, the replication process may require considerable network bandwidth. For example, to move public folders from one server to another, you must create new replicas on the server to which you want to move the public folders, wait for the hierarchy changes to replicate to the original server, and then wait for the content to replicate to the new replicas. After the replicas are synchronized, you must remove the replicas from the old server. Even removing the replicas from the old server generates network traffic because removing replicas must replicate as a hierarchy change. To learn more about how these changes to the public folder hierarchy can affect your system, see "Status Requests and Status Messages" later in this topic.
Basic Hierarchy and Content Replication Process
The following figure and the explanatory text that follows describe the basic process by which public folder hierarchy and public folder content are replicated.
.gif "The basic process public folder replication")
The details of the process are as follows:
A user modifies the public folder.
The local public folder database records the change.
At the next scheduled replication cycle (which is determined by the replication interval that you set for the public folder database), the public folder database checks the folder properties to determine which other servers contain a replica of that folder. If other replicas exist, the database determines what information must be replicated to them. This information becomes the update to the replicas.
Public folder replication is object-based. If one property of an object is modified, the entire object must be replicated. Because the database that replicates the change cannot assume that all of the receiving replicas are up-to-date, it must replicate the entire object. The implications for the different types of replication are as follows:
Hierarchy replication The replication of new hierarchy changes occurs when a public folder is created or deleted, or when a change is made to the properties of a public folder (such as a change to the replica list, client permissions, description, administrative note, or storage limits).
Content replication If a new message is posted or an existing message is modified, the update includes the entire message and its properties.
The public folder database assigns a change number to the update.
When a folder replicates an update to another server, the change number is included with the update. The receiving server then uses the change number to determine whether the update represents a new change, and whether the server is missing any data.
The public folder database packs updates into a replication message. The change numbers of all of the updates in the message are called a change number set (CNSet*)*.
Along with the updates, the public folder database packs information from the folder's entries that are in the Replication State Table, including the CNSets that were previously applied to the replica. For more information about the Replication State Table, see "Replication State Table" later in this topic.
To reduce mail traffic, the public folder database packs multiple hierarchy updates into a single replication message. Likewise, the database packs multiple content updates for the same folder into a single replication message. However, the database cannot pack hierarchy updates into the same replication message as content updates, and each content replication message contains updates for a single folder.
The public folder database addresses the replication message to the other public folder databases that host replicas of the updated folder. The database sends the message, along with any other messages that have been packed since the previous replication cycle.
To deliver replication messages, the public folder database relies on the internal routing components in Exchange. The database does not attempt to split replication messages based on topology details. If the contents of a folder are modified and the folder has five other replicas, the database generates a single replication message and addresses it to all five databases that host those replicas. The routing components determine how to route and deliver the message.
The public folder database receives the replication message.
The receiving public folder database unpacks the update and status information from the replication message.
The database compares the change numbers of the new updates to the list of change numbers that it already contains, and then identifies which updates it has not previously received.
The database applies the new updates to the appropriate folder replicas.
For each updated replica, the database updates the Replication State Table with the change numbers of the current update and the folder status information from the replication message.
If the information in the Replication State Table indicates that other CNSets have been applied to other replicas of the folder but not to replicas on this database, the database records the CNSets that are missing in a location called the backfill array, and then prepares to send a backfill request. For more information about backfill requests, see "Backfill Requests and Backfill Messages" later in this topic.
Replication Messages
The replication process uses the Active Directory attributes of the public folder databases, not the Active Directory attributes of individual public folders. The Active Directory attributes for individual public folders are used only to send regular e-mail messages to or from the folders. A public folder database object is configured and maintained automatically and resides in the Configuration container in Active Directory.
Replication messages differ from other e-mail messages in that Exchange treats replication messages as system messages. This means that replication messages are not bound by the restrictions that are applied to user e-mail messages (such as size and delivery restrictions).
The following table lists the different types of replication messages that Exchange uses.
Note
The values that are listed in parentheses in the following table are the hexadecimal notation of the message types. These notations are used in events and logs. You can use the hexadecimal value to troubleshoot replication issues.
Types of public folder replication messages and when they are used
| Message type* | When used |
|---|---|
Hierarchy (0x2) |
Replicates hierarchy changes from the local public folder database to all other public folder databases that support the same hierarchy. Although Exchange handles hierarchy changes separately from changes to content replicas, it treats the hierarchy as if it was another folder. In some event messages and other operations, Exchange refers to the hierarchy as Folder 1-1. |
Content (0x4) |
Replicates content changes from one replica to all other content replicas of that folder. A content message only contains information that applies to a single folder. |
Backfill request (0x8) |
Requests missing data in CNSets from another database. This includes hierarchy and content change numbers. |
Backfill response (0x80000002 or 0x80000004) |
Sends missing data in CNSets to a database that requested missed updates. |
Status (0x10) |
Sends the current CNSet of a folder to one or more replicas of that folder. This includes hierarchy and content change numbers. |
Status request (0x20) |
Requests CNSets to be replicated or status messages to be returned. This includes hierarchy and content change numbers. |
Replication State Table
Each public folder database maintains a Replication State Table to track the status of each replica in the database. The Replication State Table stores the following information:
Basic information that is required to construct updates to each replica.
Information about the last update to each replica that originated in the local database, including the change number of the update.
Groups of updates that have been applied to all other known replicas of the folder. Change numbers identify the updates in each group. The set of change numbers for all updates in a group is called a CNSet. Update information is passed from one database to another as part of the replication process.
The following tables provide an example of how Replication State Tables function. In this example, the public folder databases on Server A and Server B both have replicas of a folder named Projects. On each server, the Replication State Table tracks not only the status of the replica on that server, but also the status of the replica on the other server. Using this information, Server A determines whether its replica of Projects folder is synchronized with the replica of the Projects folder on Server B. Server B can likewise track its status relative to Server A.
Sample data from the Replication State Table for Server A
| Replica | Data |
|---|---|
The Projects folder on Server A (local replica) |
Last update sent: A-100 |
The Projects folder on Server B |
A-100 received B-50 received |
Sample data from the Replication State Table for Server B
| Replica | Data |
|---|---|
The Projects folder on Server A |
A-100 received B-50 received |
The Projects folder on Server B (local replica) |
Last update sent: B-50 |
By combining the lists of public folder databases that contain content replicas with the information in the Replication State Table, each public folder database can determine how up-to-date it is compared to the other public folder databases that support the public folder tree.
Backfill Requests and Backfill Messages
Backfilling occurs when a public folder database determines that it has not received all the updates for a replicated folder (or for the hierarchy) and must therefore retrieve the missing updates from another public folder database.
To streamline the backfill process, Exchange 2007 stores information about missing updates in the backfill array.
The following events may alert a public folder database to missing updates that need to be backfilled:
The status information in an incoming replication message indicates that the replica on the public folder database that sent the message has updates that are missing on the receiving database. The receiving database identifies the missing change numbers and stores them in its backfill array.
A public folder database starts for the first time. The new database sends status requests to get information about the other databases in the hierarchy. After the corresponding status messages arrive, the database populates its Replication State Table and, if necessary, the backfill array. The backfill array may contain entries for both the hierarchy and for any content replicas that the database must host.
An incoming hierarchy message indicates that a new content replica is to be placed in the public folder database. The new database sends status requests to get information about content that might be available for this replica in the other databases in the hierarchy. After the corresponding status messages arrive, the database populates the Replication State Table and, if necessary, the backfill array.
The backfill array stores this information for a specified length of time (called the backfill time-out). If the missing updates arrive in subsequent replication messages during this time, they are removed from the backfill array. The following table lists the default backfill time-out values, which depend on where the missing updates exist and whether they were previously requested.
Default time-outs used for backfill requests
| Type of request | Content exists on a database in the local Active Directory site | Content exists on a database in a remote Active Directory site |
|---|---|---|
Initial backfill |
6 hours |
12 hours |
First backfill retry |
12 hours |
24 hours |
Subsequent backfill retries |
24 hours |
48 hours |
If the backfill time-out expires, and the updates are still missing, Exchange creates one or more backfill requests and determines which servers to use as backfill sources.
To select servers to use as a backfill source, Exchange first creates a list of all the servers that have replicas of the folder and then sorts the list according to the following sequence of criteria:
Sort according to server status. Servers that are down or unavailable drop to the end of the list.
Sort according to preferred backfill server (if any). Exchange checks the public folder database object in Active Directory for a preferred backfill server. This setting is seldom used. In most circumstances, the backfill process operates most efficiently if Exchange selects a backfill server automatically. Most deployments of Exchange do not need a preferred backfill server. Microsoft Product Support Services can provide a script that sets a preferred backfill server if your deployment requires it.
Sort according to transport cost (lowest to highest). Servers in the same routing group have priority over servers in remote Active Directory sites.
Sort according to Exchange version (newest to oldest).
Sort according to the number of necessary changes that are available on the server (largest to smallest). Servers that do not have any of the missing changes are dropped from the list.
If one server does not have all the necessary changes, Exchange selects the next server in the sorted list and sends a backfill request to that server as well. This process is repeated until all of the changes are requested.
If the selected server does not respond to the backfill request, the database marks that server as unavailable and repeats the selection process. Servers that are marked as unavailable move to the end of the list.
Status Requests and Status Messages
In addition to the status information in each replication message, Exchange uses status requests and status messages to determine whether public folders must issue backfill requests.
A public folder database sends a status request under the following circumstances:
The database is notified of a change to the list of databases that hold replicas of a folder. For example, if you add a database to the list or remove a database from the list, Exchange replicates this change by using hierarchy update messages. In this case, the database sends a status request that requires every database that contains a replica of the folder to respond.
A new database has started for the first time. In this case, the database requests the status of the public folder hierarchy. The database sends a status request that requires every database that supports the public folder tree to respond.
A database that has been restored by using the Microsoft Windows Backup tool starts for the first time after the restore completes. In this case, the database requests the status of the public folder hierarchy and all of the folders for which the database contains content replicas. This status request lists two or three databases as required responders. Required responders are databases that support this hierarchy and, according to an internal selection process, are dependable sources of folder content.
To indicate the current state of a particular folder on the sending database, the public folder database sends a status message to another database under the following circumstances:
In response to a status request that is sent by another database. The status message is sent only to the requesting database and only if the following are true:
The database that received the status request is in the requests list of required responders.
The Replication State Table indicates that the database that received the status request has updates that are missing from the database that sent the request.
24 hours after the most recent update to a folder was received, if there have been no subsequent updates. Each time the database receives an update for a specific folder, the timer is reset to 24 hours. This status message is sent to the other public folder databases that contain replicas of the updated folder.
If a public folder database receives a status message indicating that the sending database has more recent information about the folder, the receiving database creates a backfill request. If the change numbers are shown to be equal (or the change numbers on the receiving server are more recent), no action is taken. For example, when a new public folder database starts for the first time, it sends status request messages to each database that supports the public folder hierarchy. Each database responds with information about the status of the hierarchy (as tracked by that database). The new database uses this information to identify which replicas (if any) it should have. The new database can then send backfill requests as needed to fill in the replica content.
Examples of Replication Cycles
The following figure illustrates a simplified two-server scenario that shows the sequence of events that you trigger when you add a content replica to a public folder database. This action adds the public folder database to the folder's replica list. Note that the sequence of steps depends on factors such as the timing of the replication intervals and the routing topology.
Sequence of events when you add a replica to a public folder database
.gif "add public folder replica to hierarchy")
The details of the process are as follows:
Working on ExServ01, an administrator adds ExServ01 to a folder's replica list.
ExServ01 sends a hierarchy message.
ExServ02 adds ExServ01 to the local copy of the folder's replica list.
ExServ01 sends a status request to ExServ02.
ExServ02 sends a status message to ExServ01 that includes the full CNSet of the folder.
ExServ01 determines that all the folder content is missing and records the appropriate entries in the backfill array.
If the content is still missing when the backfill time-out elapses, ExServ01 creates a backfill request and sends it to ExServ02.
ExServ02 compiles the content messages and sends them to ExServ01.
ExServ01 uses the incoming content messages to update the folder content and related tracking information.
If change numbers still appear to be missing, ExServ01 waits 24 hours, and then sends an updated backfill request. If a server other than ExServ02 is available, ExServ01 might send the request to that server.
The following figure illustrates a simplified two-server scenario that shows the sequence of events that you trigger when you remove a replica from a public folder database. (This action removes the public folder database from the folder's replica list.) Note that the sequence of steps depends on factors such as the number of servers in the topology.
Sequence of events when you remove a replica from a public folder database
.gif "Deleting replica from public folder database")
The details of the process are as follows:
Working on ExServ01, an administrator removes ExServ01 from a folder's replica list.
ExServ01 marks its replica (the copy of the folder on ExServ01) as delete pending.
Clients can no longer access the folder by using this database.
ExServ01 sends a hierarchy message.
ExServ02 updates its copy of the folder's replica list to show that the folder is in the delete pending state on ExServ01.
ExServ02 no longer refers clients that are looking for this folder to ExServ01.
ExServ01 sends a status request to ExServ02.
ExServ02 sends a status message to ExServ01. If the replica on ExServ02 is not up to date, ExServ02 places the appropriate entries in the backfill array. Within five minutes, ExServ02 sends the corresponding backfill request to ExServ01.
ExServ01 checks that the folder replica on ExServ02 contains all of the information that the delete pending replica does. If it does not, ExServ01 sends the appropriate content updates and returns to Step 5. Otherwise, ExServ01 continues to Step 8.
This process ensures that as long as other replicas exist, deleting a single replica does not result in a loss of content.
ExServ01 marks its replica as "delete now." The next maintenance cycle will remove the replica from ExServ01.
ExServ01 sends a hierarchy message.
ExServ02 removes ExServ01 from its copy of the folder's replica list.
Best Practices for Implementing Replication
Public folder replication in Exchange can be a resource-intensive operation. Replication requires network, CPU, and disk resources to operate. By implementing a solution that enables efficient public folder replication, especially in organizations with heavy public folder usage, you may greatly improve network, CPU, and disk load in your Exchange organization.
Generally, it is a best practice to minimize replication across the organization. By minimizing replication, you minimize the amount of data that travels over your network. Additionally, by minimizing replication, you can help make sure that multiple users are less likely to access different versions of data on multiple replicas. However, you should note that by minimizing replication, you decrease availability of the public folder data because fewer replicas of the folder are available to clients if a public folder database fails. If availability on a large scale is required for data in a specific public folder, you may require more replication.
Public Folder Replication and Continuous Replication
Continuous replication and public folder replication are two very different forms of replication that are built into Exchange. Due to interoperability limitations between continuous replication and public folder replication, there is specific guidance for combining public folder replication with continuous replication. To review this guidance, see Planning for Cluster Continuous Replication, Planning for Local Continuous Replication and Standby Continuous Replication.
For More Information
To learn more about public folders, see Understanding Public Folders.
For more information about how to manage public folders, see Managing Public Folders.
For more information about how to manage public folder databases, see Managing Public Folder Databases.