Usage of Repadmin When Troubleshooting Event ID 1311

Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2, Windows Server 2008, Windows Server 2008 R2

By all means, this topic is not about how to troubleshoot events that have Event ID 1311. In this topic, we are attempting to expose the various usage of repadmin while troubleshooting 1311 in Windows 2000 domains based on Microsoft Knowledge Base (KB) article 307593, How to Troubleshoot Event ID 1311 Messages on a Windows 2000 Professional Domain (https://go.microsoft.com/fwlink/?LinkId=121799). Some or all of the repadmin subcommands used here may be used in Windows Server 2003 environments as well.

The KB article RESOLUTION section has the following action plan. This topic examines how to apply the various repadmin subcommands against each action plan. All of the repadmin subcommands listed in this topic have associated examples either in this section or elsewhere in this document.

Resolution steps from the KB article	Action plan by using repadmin
Determine if the event ID 1311 messages are site-specific or forest-wide.	To determine the scope of event ID 1311 messages: First, find all the Inter Site Topology Generators (ISTG) in the forest. Then, examine the Directory Service logs of all the ISTG domain controllers in the forest. To determine the ISTG’s, use Repadmin /ISTG.
Determine if site link bridging is turned on and if the network is fully routed.	To determine this, use repadmin /showattr (Determine if site link bridging is turned on).
Verify that all of the sites are defined in site links.	Every site defined in Active Directory must be hosted or reside in a site link. The repadmin /showism command (Verify inter-site cost matrix and orphaned sites) is useful for locating improperly configured sites.
Detect and remove preferred bridgeheads.	To search for preferred bridgehead servers use repadmin /showattr (Determine if site link bridging is turned on).
Resolve Active Directory replication failures in the forest	When you want to discover and troubleshoot replication failures, the following repadmin subcommands can be useful: repadmin /replsummary (Monitor Forest-Wide Replication) repadmin /showrepl (Display Replication Partners and Status of a Domain Controller) Repadmin /failcache repadmin /removelingeringobjects (Windows Server 2003 only) Repadmin /KCC
Determine if source servers are overloaded.	A domain controller that is overloaded with a large number of direct replication partners or a replication schedule that is overly aggressive can create a backlog in which some partners never receive changes from a hub domain controller. The following subcommands can be useful in this situation: repadmin /showrepl (Display Replication Partners and Status of a Domain Controller) Repadmin /queue repadmin /showctx (Open sessions with the domain controller)
Determine if site links are disjointed.	"Disjoint site links" is an Active Directory configuration in which the topology is broken into two or more parts in which some sites do not replicate because site definitions and site link definitions are incorrect. Disjoint site links are the most difficult improper configuration to troubleshoot. The following subcommands may be useful in this situation: Repadmin /querysites repadmin /showconn (Can I Look at My Connection Objects and Schedule Details?) Repadmin /KCC repadmin /showrepl (Display Replication Partners and Status of a Domain Controller)
Delete connections if the KCC is in "Keep Connection Mode."	If the Knowledge Consistency Checker (KCC) builds a different path around a site-to-site connection failure, but it retries the failing connection every 15 minutes because it is in "connection keeping mode," delete all broken connections and let the KCC rebuild them. Wait two times the longest replication schedule in the forest.

Determine if site link bridging is turned on

Site link bridging is enabled in Active Directory if the following conditions are true:

• The Bridge all site links check box is selected for the IP protocol and the SMTP protocol in the Active Directory Sites and Services snap-in.

• The Options attribute for the IP protocol and the SMTP protocol is NULL or set to zero (0) for the following distinguished name (DN) paths:

  CN=IP,CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=root domain of forest

  CN=SMTP,CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=root domain of forest

There are two values that we could set from the graphical user interface (GUI): Ignore Schedules and Bridge all site links. In our example (figure 3.11.1), the IP transport has Bridge all site links enabled and SMTP transport has both values selected.

The following table lists the various values that the options attribute take.

Option value	Description
0X0	Only Bridge all site links is selected from the above
0X1	Both the values are selected
0X2	None selected
0X3	Only Ignore schedules is selected

Detect preferred bridgeheads

Preferred bridgeheads are selected when the following condition is true:

bridgeheadTransportList attribute is set to either one of the following values or both values:

• CN=IP,CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=<root domain of forest>

• CN=SMTP,CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=<root domain of forest>

By using repadmin /showattr, we are setting the base at configuration partition and applying a filter for server ObjectClass and looking for all of the domain controllers that have this value set to use either IP or SMTP transports.

If the search returns any results, note the name of server in the distinguished name path in which the bridgeheadTransportList attribute is populated.

In the example in figure 3.11.2, ROOTDC01 is selected as a preferred Bridgehead for IP transport in site HUB.

Verify inter-site cost matrix and orphaned sites

Repadmin /showism displays intersite messaging routes calculated by the Intersite Messaging Service and is very useful for locating improperly configured sites. This operation cannot be executed remotely.

As the KCC runs through the progressions of analyzing intersite site links and connections, it must query the Intersite Messaging Service (ISM) service to retrieve data about the network configuration to make intelligent decisions about routing changes.

To display cost and frequency configurations of replication between sites, use the following command:

Syntax

repadmin /showism [<TransportDN>] [/verbose]

Parameter	Description
<TransportDN>	Specifies whether the mail server is using SMTP or remote procedure call (RPC) to send messages.
/verbose	Lists detailed information.

Note

The repadmin.exe /showism cannot be executed against a remote domain controller.

Example: Display inter-site cost matrix

How to interpret the data

• Showism was used against the IP transport and hence the output is specific to IP.

• If a specific transport is not specified, the output will contain both IP and SMTP details.

• The numbers in an entry appear in the following order:

  • Cost: Replication interval: Options

• There are four key pieces of information:

  • Text regarding the status of bridgehead servers.

  • Total cost between two sites. The cost value indicates the preference for a network link for replicating directory information between sites.

  • Frequency of replication in minutes between the two sites.

  • Options for each replication link.

• In the example in figure 3.11.3, we have five sites and Bridge all site links is enabled, which means that site link transitivity is enabled. Therefore, if we see any "-1:0:0" entries for one or more covered Active Directory sites, we must ensure that the affected sites are listed in a site link. In this example, site Branch4 is not included in any site links and therefore disconnected from rest of the sites. Event 1311 will certainly occur here due to this configuration problem.

Fields of interest	Definition
"0:0:0"	Each site matrix contains one "0:0:0" entry that refers to itself.
"200:30:1"	An entry that contains positive numbers for the cost value and replication interval value (for example, "200:30:1" or "100:15:1") indicates that the site connection is good. Specifically in our example for Site BRANCH1 Site(0) CN=BRANCH1,CN=Sites,CN=Configuration,DC=contoso,DC=com 0:0:0, 200:30:1, 200:30:1, -1:0:0, 100:15:1 200 stands for the cost to replicate from site(1) which is BRANCH2 that is an aggregate cost between two hops (100 + 100) because a direct replication link between the two sites does not exist. 30 is the replication interval that is common between the two branches 1 is the option on the site link which denotes “Change Notifications are enabled across the site link” And so on the rest of the sites…
"-1:0:0"	A "-1:0:0" entry indicates that the site connection is not working. This occurs if one or more of the following conditions are true: Site is not included in a site link. Site does not host any domain controllers (this is known as an "uncovered" site). Replication protocol is not used. For example, if SMTP replication is not configured, the entries in the SMTP portion of the /SHOWISM matrix all appear as "-1:0:0".

Note

If site link bridging is enabled and the repadmin /showism command returns a site with a full complement of "-1:0:0" entries and one "0:0:0" entry is orphaned unless the site is uncovered (no domain controllers reside in that site).

If site link bridging is disabled, "-1:0:0" entries are less meaningful. If this is the case, you must manually determine if each site is included in a site link. To do so, write down the list of sites and site links, and manually map each site to a site link.

Repadmin /failcache

Repadmin /failcache displays a list of replication failures that KCC is aware of. Run this command from the console of each ISTG domain controller in the forest to discover replication failures for bridgeheads in the site for that ISTG.

Syntax

repadmin /failcache <DC_LIST>

Parameter	Description
<DC_LIST>	Specifies the host name of a domain controller, or a list of domain controllers separated by a space. For details about <DC_LIST>, see repadmin /listhelp.

Example: Display replication failures that KCC is aware of

The example in figure 3.11.4 shows sample output from the repadmin /failcache command.

The output from the repadmin /failcache command is divided into two sections explained in the following table.

KCC Link Failures

Lists errors for existing connection links. The ISTG domain controller imports showreps ("repsfroms") data for every bridgehead server in its site. However, the ISTG domain controller does not list errors. The link failure cache is emptied at the beginning of every KCC run and refilled during the course of the current run.

KCC Connection Failures

Lists unsuccessful attempts to build connection objects between domain controllers ("reps from" or "reps to"). When you run the repadmin /failcache command from the ISTG domain controller, it lists entries that are imported from bridgeheads in the site. At the beginning of each KCC run, the KCC examines each entry in the connection failure cache and tries to DsBind to the failing server. If the bind succeeds, the entry is removed.

In the example in figure 3.11.4.1, the failures are a result of some topology changes from the past and would continue to exist due to the value of the replTopologyStayOfExecution attribute, which determines how long domain controller metadata is retained in Active Directory after a domain controller has been removed.

Example: Output when there are no failures

When there are no failures, the output should appears as it does in figure 3.11.4.2.

Note

The repadmin /failcache command differs from the repadmin /showrepl command in two ways:

• The repadmin /showrepl command shows the naming context that is failing.
  
  
• The repadmin /failcache command does not.
  
  

Repadmin /KCC

Repadmin /KCC forces the KCC to recalculate replication topology for a specified domain controller. By default, this recalculation occurs every 15 minutes.

Syntax

repadmin /kcc <DC_LIST> [/async]

Parameter	Description
<DC_LIST>	Specifies the host name of a domain controller, or a list of domain controllers separated by a space. For details about <DC_LIST>, see repadmin /listhelp.
/async	Specifies that replication will be asynchronous. This means that repadmin starts the replication event, but it does not expect an immediate response from the destination domain controller. Use this parameter to start the KCC and not wait for it to finish.

Example 1: Running the KCC on the local domain controller

Example 2: Running the KCC against the ISTG of the HUB site

Example 3: Running the KCC against all the global catalog servers in the forest

Example 4: Running the KCC against all the domain controllers in the BRANCH2 site

Repadmin /ISTG

Repadmin /ISTG returns the server name of the ISTG server for a specified site.

Syntax

repadmin /istg <DC_LIST> [/verbose]

Parameters	Descriptions
<DC_LIST>	Specifies the host name of a domain controller, or a list of domain controllers separated by a space. For details about <DC_LIST>, see repadmin /listhelp.
/verbose	Lists detailed information.

Example: Display ISTGs in my environment

In the example in figure 3.11.6, the ISTGs are listed from the perspective of the local domain controller from which the command was run. It is important to note that this information may be different from the perspective of each domain controller, depending on the forest-wide Active Directory convergence time and replication status.

Repadmin /querysites

Repadmin /querysites use routing information to determine cost of a route from a specified site to another specified site or sites.

Syntax

repadmin /querysites <FromSiteRDN> <ToSite1RDN> <ToSite2RDN>...]

Parameter	Description
<FromSiteRDN>	Specifies the relative distinguished name of the site from which the cost is calculated.
<ToSite1RDN>	Specifies the relative distinguished name of the site to which the cost is calculated.

Example 1: Display cost between BRANCH1 and HUB

Example 2: Display cost between BRANCH1 and BRANCH2

Due to site link transitivity, the cost from BRANCH1 to BRANCH2 is aggregated by adding the cost from BRANCH1 to HUB (100) with the cost from HUB to BRANCH2 (100).

Example 3: Display cost between BRANCH1 and Branch2

Note that the relative distinguished name of the site is case sensitive and hence the error.

Note

The relative distinguished name of the site is case sensitive.

The repadmin /querysites parameter does not allow the use of alternate credentials.

Repadmin /queue

Repadmin /queue displays tasks that are waiting in the replication queue.

Syntax

repadmin /queue <DC_LIST>

Parameter	Description
<DC_LIST>	Specifies the host name of a domain controller, or a list of domain controllers separated by a space. For details about <DC_LIST>, see repadmin /listhelp.

Example: Display the queue length against the local domain controller

Under normal circumstances this list should always be empty and the command should be run outside of the replication window when troubleshooting domain controller overload was caused due to replication requests.

Example: Queue contains one item

Repadmin /bridgeheads

Repadmin /bridgeheads lists the bridgehead servers for a specified site.

Syntax

repadmin /bridgeheads [<DC_LIST>] [/verbose]

Parameter	Description
<DC_LIST>	Specifies the host name of a domain controller, or a list of domain controllers separated by a space. For details about <DC_LIST>, see repadmin /listhelp.
/verbose	Lists detailed information.

For clarity:

• The following example shows only bridgeheads only for the HUB site.

• The following example shows the normal and verbose modes to help compare them.

• “The RPC service is unavailable” status is abbreviated as RPC.

• “The operation completed successfully” status is abbreviated as status.

Example 1: Repadmin /bridgeheads rootdns

Bridgeheads for site HUB (rootdns.contoso.com):
Source Site    Local Bridge  Trns         Fail. Time    #    Status
===========    ============  ====     ==============   ===   ======
BRANCH2       BRANCH-HUB-BH   IP 2005-02-14 14:18:52    3    RPC.
Configuration research
BRANCH1       BRANCH-HUB-BH   IP             (never)    0    Success.
Configuration ForestDnsZones DomainDnsZones research
BRANCH3       BRANCH-HUB-BH   IP             (never)    0    Success.
Configuration DomainDnsZones ForestDnsZones research

Example 2: Repadmin /bridgeheads rootdns /verbose

Bridgeheads for site HUB (rootdns.contoso.com):
Source Site    Local Bridge  Trns         Fail. Time    #    Status
===========    ============  ====     ==============   ===   ======
BRANCH2        BRANCH-HUB-BH  IP  2005-02-14 14:18:52   3    RPC.
Naming Context    Attempt Time         Success Time  #Fail  Last Result
==============    ============         ============  =====  ===========
Configuration  2005-02-14 14:51:41  2005-02-14 14:18:51  3   RPC.
research       2005-02-14 14:53:15  2005-02-14 14:18:52  2   RPC.
Source Site    Local Bridge  Trns         Fail. Time    #    Status
===========    ============  ====     ==============   ===   ======
BRANCH1        BRANCH-HUB-BH  IP             (never)    0   Success
Naming Context    Attempt Time         Success Time  #Fail  Last Result
==============    ============         ============  =====  ===========
Configuration  2005-02-14 14:51:41  2005-02-14 14:51:41  0   Success.
ForestDnsZones 2005-02-14 14:52:37  2005-02-14 14:52:37  0   Success.
DomainDnsZones 2005-02-14 14:53:15  2005-02-14 14:53:15  0   Success.
research       2005-02-14 14:52:37  2005-02-14 14:52:37  0   Success.
Source Site    Local Bridge  Trns         Fail. Time    #    Status
===========    ============  ====     ==============   ===   ======
BRANCH3        BRANCH-HUB-BH  IP             (never)    0    Success.
Naming Context    Attempt Time         Success Time  #Fail  Last Result
==============    ============         ============  =====  ===========
Configuration  2005-02-14 14:51:42  2005-02-14 14:51:42  0   Success.
DomainDnsZones 2005-02-14 14:53:15  2005-02-14 14:53:15  0   Success.
ForestDnsZones 2005-02-14 14:52:37  2005-02-14 14:52:37  0   Success.
research       2005-02-14 14:53:15  2005-02-14 14:53:15  0   Success.

How to interpret the data

Repadmin /bridgeheads is run remotely against a domain controller in the HUB site and the output is the perspective of the topology for ROOTDNS. In these examples, we are seeing local bridgehead server BRANCH-HUB-BH is having replication problems with the remote bridgehead server in the BRANCH2 site.

Fields of interest	Explanation
Source Site	Source site from where the local bridge head (inbound) is pulling data. Remember replication is always inbound.
Local Bridge	Local Bridge head server for the site for which the tool is displaying results. In the example in figure 3.11.9.2, BRANCH-HUB-BH is the bridgehead server of the HUB site.
Trns	In the example in figure 3.11.9.2, the transport is IP.
Fail time	This is the last successful replication time.
#	Number of failures since the last successful replication time.
Status	Replication status.
Naming Context	Directory partition. Remember Bridgeheads are partition specific.
Attempt time	Last replication attempt time with the remote bridgehead.
Success time	Last successful replication time with the remote bridgehead.
#Fail	Number of attempts since the failure per partition.
Last result	Latest replication status.

Note

Replication is performed for each partition. But sometimes we do not see the Schema partition listed in the previous example as a naming context (partition) and hence there are no bridgeheads listed. This is not a limitation of the tool; it has to do with the how information is stored in the connection object that is queried to determine the bridgehead. If you see the configuration partition in the output, it is implied that schema is also included because the KCC calculates the configuration and schema partitions to have the same replication topology.

Repadmin /showmsg

Repadmin /showmsg displays the error message for a given error number.

Syntax

repadmin /showmsg <Win32Error> | <DSEventID> /NTDSMSG}

Parameter	Description
<Win32Error>	Returns a short description of the given Win32 error code.
<DSEventID>/NTDSMSG	Returns the actual event log text for the specified event ID.

Example: Display the error message for the win32error 1722 and DS event ID 1404

Repadmin /viewlist

By default, this subcommand is used to displays a list of domain controllers. It could also be used to form an Lightweight Directory Access Protocol (LDAP) query to list only objects in the directory.

Syntax

repadmin /viewlist <DC_LIST> <OBJ_LIST>

Parameter	Description
<DC_LIST>	Specifies the host name of a domain controller, or a list of domain controllers separated by a space. For details about <DC_LIST>, see repadmin /listhelp.
<OBJ_LIST>	This parameter takes a distinguished name (DN) or a special keyword that expands into a DN. The keywords are: Ncobj:config: This keyword is the Configuration directory partition for the forest. Ncobj:schema: This keyword is the Schema directory partition for the forest. Ncobj:domain: This keyword is the domain partition DN of the home server. Dsaobj: This keyword is the NTDS settings object of the home server.

Example 1: Display all the DC’s in the forest

Example 2: Display all the Group Policy objects in the domain directory partition for the domain of the domain controller that repadmin is running against

Note the usage of OBJ_LIST and OBJ_LIST OPTIONS. For details please refer to repadmin /listhelp.

Open sessions with the domain controller

The repadmin /showctx command displays a list of computers that have opened sessions with a specified domain controller.

Syntax

repadmin /showctx <DC_LIST> [/nocache]

Parameter	Description
<DC_LIST>	Specifies the host name of a domain controller, or a list of domain controllers separated by a space. For details about <DC_LIST>, see repadmin /listhelp.
/nocache	Specifies that globally unique identifiers (GUIDs) are left in hexadecimal form. By default, GUIDs are translated into strings.

Example: Show open sessions with a DSA