It is a complex environment with 15 domains,30+ sites and 130+ DCs where power failures and network related issues frequently disrupt AD operations.

So I wanted to share with you the method I used to remove lingering objects and hopefully you find it useful somehow.

First download repldiag from https://activedirectoryutils.codeplex.com/releases/view/13664 as it will save you a lot of typing.

Then create a .bat file similar to the following:

/enable Strict Replication Consistency across the forest

repadmin /regkey * +strict

/dump all repldiag commands to a .txt file (repldiag enumerates all domains, finds all DCs and creates all necessary repadmin commands for removing lingering objects)

repldiag /removelingeringobjects /outputrepadmincommandlinesyntax >output.txt

/dump domain specific repldiag commands to a .bat file (this way you may cleanup one domain at a time)

findstr "domain_a" output.txt >domain_a.bat
(...)
findstr "domain_n" output.txt >domain_n.bat

/call all .bat files in order to remove lingering objects

call domain_a.bat
(...)
call domain_n.bat

/Then run repadmin /showrepl against all DCs in each domain and pipe it to a .txt file

repadmin /showrepl *.domain_a /errorsonly > domain_a.txt
(...)
repadmin /showrepl *.domain_n /errorsonly> domain_n.txt

/Look for "failed, result 8606 (0x219e): Insufficient attributes were given to create an object. This object may not exist because it may have been deleted and already garbage collected."

/Identify the destination DC (the DC logging the event), Source DC (the DC that contains lingering objects) and affected Naming Context (the NC that contains lingering objects)

/With the information above force replication to resume on the affected DCs/NCs

repadmin /replicate destination sourceGUID NC /force

as an example:

**output from showrepl**

Repadmin: running command /SHOWREPL against full DC DC1.domain_a.fqdn

site1\DC1

DSA Options: IS_GC

Site Options: (none)

DSA object GUID: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa

DSA invocationID: zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz

==== INBOUND NEIGHBORS ======================================

DC=domain_n,DC=fqdn

    Site30\DC via RPC

        DSA object GUID: nnnnnnnn-nnnn-nnnn-nnnn-nnnnnnnnnnnn

        Last attempt @ 2014-10-16 10:55:42 failed, result 8606 (0x219e):

            Insufficient attributes were given to create an object. This object may not exist because it may have been deleted and already garbage collected.

        529 consecutive failure(s).

        Last success @ 2014-09-30 07:00:33.

In this example the command to force replication to resume would be

repadmin /replicate DC1.domain_a.fqdn  nnnnnnnn-nnnn-nnnn-nnnn-nnnnnnnnnnnn DC=domain_n,DC=fqdn /force

Hope it helps!

Paulo

Have you ever used Fine Grained Password Policies?

This feature introduced in Windows Server 2008 allows you to override password policy set at the domain level.

It applies password settings to subsets of users that you may like to differentiate from the domain policy.

In Windows Server 2012 we added a GUI (Graphical User Interface) so now you don't have to use ADSIedit, LDP or Powershell to create PSOs (Password Settings Objects).

Note that PSOs are not like GPOs:

1. They're not managed via GPMC.

2. They're not linked to OUs, Sites or Domains.

PSOs apply to User and Group objects (ie. ultimately apply to User Accounts)

As an example, with FGPP you can have a Domain password policy that defines a minimum password length of 8 characters which will be applied to all users in the domain.

Then have a PSO that sets 24 characters for all user accounts that are members of the "All Service Accounts".

I added the following video that walks you through the steps needed to implement this (once again I've kept it short and simple and no sound).

Anyway here's the high level steps you have to follow:

1. Using ADAC (Windows 8 or Server 2012) open the Password Settings container (under System container).

2. Add a New PSO (Password Settings Object).

3. Configure the desired PSO properties (Max password Age, Min Password Length, etc).

4. Assign the PSO to a user or group

Hope it helps!

Enjoy!

The AD recycle bin feature has been released on Windows Server 2008 R2 without a graphical user interface, which made it's deployment and usability (I mean recovering deleted objects from AD) somewhat difficult for system admins.

In this post I will write about how to enable the ADRB feature on both WS2008 and WS2012.

This will provide you with the skills necessary to perform object recovery regardless of the operating system you are using currently.

Be amazed on how easy it is to recover deleted objects in WS2012.

This feature can be enabled if your forest is running at WIN2008R2 functional level.

If you already have all DCs in the forest running on Windows Server 2008R2 or higher you may use the following powershell command to raise the FFL:

Set-ADForestMode 4 -Identity <forestname>

which requires that all domains in the forest run at WIN2008R2 domain functional level, so if needed run:

Set-ADDomainMode 4 -Identity <domainname>

Before running the powershell commands above and if using WS2008R2 you have to import the Active Directory module for powershell (WS2012 does it automatically).

Important: Enabling Active Directory Recycle Bin is an irreversible procedure.

To enable the AD Recycle Bin feature using powershell run:

Enable-ADOptionalFeature –Identity ‘CN=Recycle Bin Feature,CN=Optional Features,CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=contoso,DC=com’ –Scope ForestOrConfigurationSet –Target ‘<forestname>’

To check if recycle bin is enabled:

Get-ADOptionalFeature -filter *

To restore a deleted object do the following:

Get-ADObject -Filter {displayName -eq "name"} -IncludeDeletedObjects | Restore-ADObject

Watch the video - Enable AD Recycle Bin and restore a single object using powershell on WS2008R2

Although restoring a single object is fairly simple, the restoration procedures get more complex when restoring multiple nested objects.

For example when you need to restore an OU with hundreds or thousands of objects in it (like users, groups, computers or even sub OUs which in its turn may contain more objects).

You will have to investigate how deep OU nesting is by querying deleted users lastknownparent attribute in order to understand the deleted OU structure.

Then you must start restoring from top to bottom so one hierarchy level at a time.

The reason behind is that when an object is deleted and moves to the deleted objects container, the object distinguished name (DN) gets mangled and the deleted objects container doesn't contain/keep an hierarchy.

The only way to return the objects hierarchy is by searching for it's lastknownparent attribute.

For example consider the following scenario:

[picture 1]

In this scenario we need to start by searching for known objects that have been deleted, for example in this case by finding the lastknownparent of a user named Peter:

Get-ADObject -SearchBase "CN=Deleted Objects,DC=contoso,DC=com" -ldapFilter:"(msDs-lastKnownRDN=Peter)" – IncludeDeletedObjects –Properties lastKnownParent

Then by finding all objects that have HumanResources as lastknowparent (note: add another \ before 0ADEL:):

Get-ADObject –SearchBase "CN=Deleted Objects,DC=contoso,DC=com" -Filter {lastKnownParent -eq 'OU=HumanResources\ \0ADEL:c876daac-da9b-57ce-bded-978aed9c0e2b,CN=Deleted Objects,DC=contoso,DC=com'} -IncludeDeletedObjects - Properties lastKnownParent | ft

At this stage we must go through the output and look for other OUs contained within HumanResources OU, then we need to search for other deleted objects inside it in case we find extra OUs (in this case we explore the Users OU within HumanResources):

Get-ADObject –SearchBase "CN=Deleted Objects,DC=contoso,DC=com" -Filter {lastKnownParent -eq 'OU=Users\ \0ADEL:6b507c43-172b-8145-93bf-61e00302bb4a,CN=Deleted Objects,DC=contoso,DC=com'} -IncludeDeletedObjects - Properties lastKnownParent | ft

So far we found that HumanResources OU has been deleted (by noticing the mangled DN) and with the following command we may validate if there is any other OU above it and if it was also deleted (if so we have to perform the same steps as above to find other objects within it):

Get-ADObject -SearchBase "CN=Deleted Objects,DC=contoso,DC=com" -ldapFilter:"(msDs-lastKnownRDN=HumanResources)" –IncludeDeletedObjects –Properties lastKnownParent

After investigation is complete, then we can start restoring the objects from top to bottom:

to restore the HumanResources OU:

Get-ADObject -ldapFilter:"(msDS-LastKnownRDN=HumanResources)" –IncludeDeletedObjects | Restore-ADObject

to restore all deleted objects under it (including the Users OU):

Get-ADObject -SearchBase "CN=Deleted Objects,DC=contoso,DC=com" -Filter {lastKnownParent -eq "OU=HumanResources,DC=contoso,DC=com"} -IncludeDeletedObjects | Restore-ADObject

To restore all deleted objects under the Users OU:

Get-ADObject -SearchBase "CN=Deleted Objects,DC=contoso,DC=com" -Filter {lastKnownParent -eq "OU=Users,OU=HumanResources,DC=contoso,DC=com"} -IncludeDeletedObjects | Restore-ADObject

Note: Depending on your infrastructure you may have to go deeper into OU hierarchy, but then you just have to repeat the steps above accordingly.

Watch the video - Restore multiple objects using powershell on WS2008R2

Now that we covered how to restore deleted objects using the recycle bin attribute in WS2008R2 lets see how easy it is to do the same in WS2012 with the new recycle bin GUI:

You have to open the Deleted Objects container using ADAC and perform the same searches on it just by using the UI and by working out the hierarchy by looking at the Last Known Parent attribute.

If looking for specific objects we may click add criteria and in case we don't know exactly what to search for -this is one of the reasons why having AD proper documented is so important - an idea would be to add "and Last modified between these dates:")

Watch the video - Restore multiple objects using the NEW Recycle Bin GUI in WS2012

Hope it helps!

Enjoy!

PS: I will add the videos throughout the week.

In my next post I will share more information on how actually the AD recycle bin works.

On one of my previous posts I wrote about vDC cloning which is my preferred feature in Windows Server 2012 "https://blogs.technet.com/b/reference_point/archive/2012/12/11/so-you-wanted-to-deploy-domain-controllers-faster-now-you-can.aspx".

VDC cloning gives you the ability to scale up your production forest and to recover from disasters faster, or simply to build a lab in a blink of the eye.
 
In this post you may watch a number of videos that show how to clone a vDC.

Note: I tried to keep the videos as simple as possible for quick reference.

HOW TO CLONE A vDC

The Requirements are:
Hypervisor must have support for VMGID (VM-Generation ID).
ADDS schema version 56
Windows Server 2003 Forest Functional Level.
Source DC must be running Windows Server 2012.
PDCe must be running on a Windows Server 2012 DC.
PDCe and RID master online and available.

The video contents are:

Video 1 - Pre-requisit check.

Step 1 - Verify that the source VDC is running on a supported hypervisor.
Step 2 - Verify Schema version.
Step 3 - Verify Forest Functional Level.
Step 4 - Check if the VDC source Operating System.
Step 5 - Verify that the PDCe FSMO role is running on a Windows Server 2012 DC
Step 6 - Ensure that PDC and RID master are available during cloning process.

Video 2 - Getting the Clone ready.

Step 7 - Create DCCloneConfig.xml file.
Step 8 - Add the source VDC to the "Cloneable Domain Controllers" security group.
Step 9 - Shutdown the source VDC.

Video 3 - Cloning...

Step 10 - Export the source VM (Virtual Machine).
Step 11 - Import the VM with the option "Copy the virtual machine (create a new unique ID)".
Step 12 - Start the new VM.

Detailed steps:

Step 1 -  Verify that the source VDC is running on a supported hypervisor.

On the source vDC:
open [Device Manager]
expand [System Devices]
open properties of [Microsoft Hyper-V Generation Counter]
select the "Driver" tab
click "Driver details"
verify that the driver is "vmgencounter.sys"
This is the driver that makes vDC cloning and snapshot restore possible in Windows Server 2012.

Step 2 - Verify Schema version.

On any DC in the forest:
run [regedit]
browse to HKLM\System\CCS\Services\NTDS\Parameters
verify that "Schema Version" REG_DWORD value is 56.
This is the Windows Server 2012 version of the schema.

Step 3 - Verify Forest Functional Level.

On any DC in the forest:
open [Powershell]
run [Get-ADForest]
verify that "ForestMode" value is "Windows2003Forest" or higher.

Step 4 - Check the vDC source Operating System.

On the source vDC:
run [winver]
verify that source vDC is a Windows Server 2012.

Step 5 - Verify that the PDCe FSMO role is running on a Windows Server 2012 DC

On any DC in the domain:
open [cmd]
run [netdom query fsmo]
copy the PDC FQDN
open [Powershell]
run [Get-ADDomainController -server <paste the PDC FQDN here>
verify that OperatingSystemVersion value is 6.2 (9200) or higher

Step 6 - Ensure that PDC and RID master are available during cloning process.

Step 7 - Create DCCloneConfig.xml file.

open [Powershell]
run [New-ADDCCloneconfigFile]
(this will create an empty configuration file, you might want to have a look on the table below before you add
 configuration information to this file)

Note: If you get a failure for applications not cloneable you have 2 options add them to the excluded application list or uninstall them from the DC

To see the list of unsupported applications run:

Get-ADDCCloningExcludedApplicationList

and to generate the XML file (thus adding the application(s) to the excluded list run:

Get-ADDCCloningExcludedApplicationList -GenerateXML

Step 8 - Add the source VDC to the "Cloneable Domain Controllers" security group.

open [ADAC]
browse your domain to the "Users" container
double-click "Cloneable Domain Controllers" security group
Select "Members" tab and click "Add" button to add the source domain controller account

Step 9 - Shutdown the source VDC.

Step 10 - Export the source VM (Virtual Machine).

open [Hyper-V Manager]
Right-click the source vDC VM
Select Export
Specify where you want to save the files

Step 11 - Import the VM with the option "Copy the virtual machine (create a new unique ID)".

open [Hyper-V Manager]
click on "Import Virtual Machine"
Locate Folder
Select Virtual Machine
Choose import type:  "Copy the virtual machine (create a new unique ID)"

Step 12 - Start the new VM.

(Refer to the diagram below in order to understand the cloning/snapshot restore decision process)

The Cloning/Snapshot safeguards are:

 The following table puts together the outcomes of the diagram above:

By the way you may find a playlist of all 3 videos above at: 

https://www.youtube.com/playlist?list=PLRiiq9ROPBOtJhPx2SciZcMfhJ4PN4K7y 

Hope it helps!

 Best regards

Paulo

As you could guess from my previous post, Microsoft has introduced new features in Hyper-V and Windows Server 2012 that allow to do things with Virtualized DCs that before were impossible.
This has became possible due to the introduction of an identifier called VM-Generation ID on the Hypervisor driver and the corresponding attribute in AD (msDS-GenerationID).

The feature I want to tell you about is VDC Cloning.

Now a Cloned Windows Server 2012 Domain Controller automatically executes Sysprep and uses the existing NTDS and SYSVOL data for promotion (similar to IFM - Install from Media). Additionally, you may create a configuration file (DCCloneConfig.xml).
Then just need to copy or export the source VDC and it's done.
You get faster deployment, scalability and easier disaster recovery.

Security wise, Domain Admins select DCs that can be cloned.
The Hyper-V administrators can only deploy the approved cloned DCs thus ensuring that unauthorized users cannot create rogue DCs.

The following table exemplifies how VDC Cloning is achieved:

So in a nutshell cloning succeeds the following will happen:

If both IDs don't match and a DCCloneConfig.xml file exists. (If xml file doesn't exist then VDC performs Snapshot safe restore - please refer to my previous blog for more information).

DSRM boot flag is set (to prevent boot into production in case cloning fails)

Then VDC checks for VDCisCloning DWORD under NTDS parameters reg key, and if it doesn't exist NTDS invalidates RID pool and resets Invocation ID; if exists then cloning has failed before (and went to DSRM as a safety mechanism) and this a re-attempt of cloning.

The Cloned VM gets the configuration settings (from DCCloneConfig.xml) and all ADDS related services are stopped.

Time synchronization is enforced (using DOMHIER) and Kerberos tickets are purged.

All FRS/DFSR database files are deleted (Not the SYSVOL content which will be used later for pre-seeding in order to minimize SYSVOL replication traffic) and services set to start automatically.

The cloned VM is renamed and DC promotion starts using the existing NTDS.DIT file as source (which will minimize replication traffic later).

Gets a new RID pool from the RID Master.

New Invocation ID and NTDS settings are created.

Starts inbound replication.

FRS/DFSR services start and non-authoritative restore of the SYSVOL is triggered.

Enables DNS client registration.

Runs Sysprep (DefaultDCCloneAllowList.xml determines which Sysprep modules are run)

Once promotion completes the DSRM boot flag is removed, the DCCloneConfig is renamed (will prevent it from being read during reboot),

the VDCisCloning DWORD removed and sets "VDC Cloning Done" to 1.

The value of VM-GenerationID (from HV) is copied to the VDC computer account VM-Generation ID (msDS-GenerationID attribute).

Finally once the VDC reboots it starts advertising itself as a DC.

Note: Currently, this feature only works on Windows Server 2012 Hypervisor or HV2012.

Hope this helps on your future DC deployments and feel free to comment.

Thanks

Paulo

Every DC maintains a table - ReplUpToDateVector - (or Up-to-Dateness Vector) per Naming Context (NC or AD partition).
These tables record data from the local DC and its replication partners, this data includes the uuidDSA (or DSA GUID); usnHighPropUpdate (or High Watermark) and timeLastSyncSuccess (or the time stamp of last successfull replication from that replication partner) for a particular partition.

When a change is made (ie. an object is created or deleted, an attribute of an object is modified) one (or more) attribute(s) will have their Originating and Local USN incremented.

Example:

repadmin /showobjmeta * OU=USNROLLBACK,DC=contoso,DC=com

Additionally, the ReplUptoDateVector table (UTDVEC table from now on) on the local DC for itself will be updated.

screenshot of repadmin /showutdvec * dc=contoso,dc=com

Normal operation, In this case there is no outstanding replication from ContosoDC1 to ContosoDC2 (values of ContosoDC1 match on both DCs; on the other hand ContosoDC1 will have to replicate from ContosoDC2 66 changes (220356-220290)

Then a replication partner will compare its own version of the table and requests the changes that are higher than the High Watermark from the source.

If the USN for the DC on the replication partners is higher than the one the DC has for itself, you are dealing with a USN Rollback.

Example:

screenshot of repadmin /showutdvec ( relevant USNs highlighted)

USN Rollback. Note that ContosoDC1 "thinks" that ContosoDC2 has a higher "High Watermark" than in reality. So without the USN rollback protection mechanism the next 182 (220380-220198) changes originated on ContosoDC2 will be discarded by ContosoDC1

In that case the originating DC will log Replication Event ID 2095 in Directory Service log and will disable inbound and outbound replication as a protection mechanism in order to avoid further damage.

screenshot of event 2095

Without this safety valve further changes held on the originating DC will never be replicated, and eventually only when the originating DC catches-up with the USN known by its replication partners will start replicating again, however any changes in between are lost forever.

NOTE: Ensure that "Allow replication with corrupt and divergent partners" is not in use, or this protection will be ignored.
 
In order to avoid this issue you should backup your DCs using a supported method, that is an AD aware backup application (NTBackup and Windows Backup are AD aware as so other 3rd party backup applications).
What you should NOT do as a replacement for the applications above is to restore AD from unsupported backup methods like:

Disk Mirroring
Cloning
VHD copies
VM snapshots
or any other cloning method that doesn't reset the DSA Invocation ID when an AD restore is executed.

Note:

The DSA invocation ID is reset once you restore AD using a supported method. Thus replication partners will update their UTDVEC tables with the new value for the restored DC. This doesn't happen when using unsupported methods.

To fix the problem there are two supported methods:

1. Reinstall Active Directory on the affected Domain Controller.

Transfer any FSMO roles if needed.
Demote the DC.
Perform a metadata clean-up of all references to the DC.
Re-promote the DC.

2. Restore the System State.

If a valid system state backup was made before the DC was restored from one unsupported method. Restore the system state from the most recent backup.

For more information on how AD replication works and USN rollback please refer to the following articles:

How the Active Directory Replication Model Works
https://technet.microsoft.com/en-us/library/cc772726(WS.10).aspx

Running Domain Controllers in Hyper-V
https://technet.microsoft.com/en-us/library/d2cae85b-41ac-497f-8cd1-5fbaa6740ffe(v=WS.10)#usn_and_usn_rollback

In Windows Server 2012 virtualized Domain Controllers, you can now restore snapshots without permanently damage domain controllers.
While this does not prevent other issues for other technologies and applications, it does make domain controller virtualization safer.

Now virtualized domain controller snapshot restore resets the DC's unique Invocation ID.
Additionally discards the local RID pool and non-authoritatively restores the SYSVOL folder. 
This means that accidentally restoring a snapshot is no longer an unsafe operation on domain controllers.

The following process describes how Virtualized DC (VDC) Safe Restore is achieved:

1. Restore of an existing virtual machine (VM) domain controller from a snapshot in a hypervisor that supports VM-Generation ID (Windows Server 2012 Hyper-V for instance).

Assuming that this VM already has an existing VM Generation-ID on its DC computer object when the snapshot was taken as part of the msDS-GenerationID attribute (Schema version 56).

2. The VM then reads the VM-Generation ID provided by Hyper-V VMGenerationCounter driver and compares the VM-Generation IDs values.

If they do not match, it continues with snapshot restoration operations.
Once restored, the Generation-ID set on the DC computer object (in AD) is updated to match the new ID provide by the hypervisor host.

If the hypervisor does not provide a VM-Generation ID for comparison, the guest will operate like a Windows Server 2008 R2 or earlier virtualized domain controller.

3. The Virtualized DC then:

Invalidates the local RID pool.
Sets a new DSA invocation ID.

4. Non-authoritative inbound replication is triggered  from a replication partner. The DC requests changes starting at a USN that precedes the USN at which the local directory service was restored. The UTDVEC table of the destination DC is updated appropriately.

5. The virtualized DC synchronizes the SYSVOL:

If using FRS, it stops the NTFRS service and sets the BURFLAGS registry value (D2).
It then starts the NTFRS service, thus performing a non-authoritative restore of the SYSVOL.

If using DFSR, it stops the DFSR service and deletes the DFSR database files. It then starts the DFSR service, thus performing a non-authoritative restore of the SYSVOL.

6. The VM updates the msDS-GenerationID attribute on its own DC object to match the current Hypervisor VM-Generation ID

If you haven't got started with Windows Server 2012, download Windows Server 2012 from:
https://technet.microsoft.com/en-us/evalcenter/hh670538.aspx

To finish this post as a personal note, and from all improvements in Windows Server 2012 this my preferred feature. And I think that whomever imagined it deserves a big round of applause from all of us.

Feel free to leave your comments.

Cheers.

To reset the Secure Channel of a Domain Controller you have 2 methods, one requires reboot the other doesn't however requires more tasks to be completed.

If restarting the DC is an issue for you due to business, availability or change control requirements do the following:

1. Set the Kerberos Key Distribution Center (KDC) service startup type to Disabled, and restart the domain controller
(particularly important if you have more than one DC on the same domain, so this way you will force the affected DC to contact another DC for kerberos authentication, instead of using itself)

2. Once the DC restarts, reset the secure channel:

netdom resetpwd /server: /userd:\ /passwordd:*
(you should get the foolowing message: "The machine account password for the local machine has been successfully reset")

3. Then, set the startup type for the KDC service back to Automatic and restart the DC once again.

Note: Do not start the service manually as it will be started automatically during reboot.

The 2nd way is:

1. Delete any mapped drives that might exist on the affected DC, for that run:

net use * /d

2. Stop the KDC service.

3. Purge all kerberos tickets from the computer credentials cache (this is the memory space were the computer stores its tickets, there's nothing in common with cached credentials).

Depending on the Operating System run the following:

On Windows Server 2003:

at /interactive cmd.exe
(check current system time and add a couple of extra minutes, for example if current time is 14:14 type 14:16)

Note:
Another command prompt will launch under the System account context (SVCHost.exe )on the Console session of the problem DC at the time you entered (14:16 in the example above).
If running the command via RDP ensure that you are connected to session 0 (mstsc /console or /admin depending on mstsc version) or you won't be able to see the 2nd command prompt.

klist purge

On Windows Server 2008 or above:

klist -li 0x3e7 purge

Note:

The -li 0x3e7 gets klist into the computer credentials cache so there's no need for the "at" command.

4. Reset the Secure Channel:

netdom resetpwd /server: /userd:\ /passwordd:*
(You should get the following message "The machine account password for the local machine has been successfully reset"

 
net use \\\IPC$
(The DC will request new kerberos tickets)

5. Force the DC to replicate from/to the PDCemulator (so both ways first inbound then outbound):

repadmin /replicate <destination> <source> <NC> /force

example:
repadmin /replicate DC1 DC2 DC=Contoso,DC=com /force
(you should get the following message "sync from DC2 (the PDCe) to DC1 (the affected DC) completed successfully")
repadmin /replicate DC2 DC1 DC=Contoso,DC=com /force
(you should get the following message "sync from DC1 (the affected DC) to DC2 (the PDCe) completed successfully")

Note:
If the affected DC successfully replicates from the PDCe then start the KDC service. If not successful, you will have to use the method the requires restart.
If replication to the PDCe fails, delete the old outbound connection object and trigger KCC (repadmin /kcc) in order to rebuild them, and force outbound replication again. (this might be required for COs with other replication partners)

Done.

In order to verify the health status of secure channels you may use Nltest.exe or Netdom.exe (which is also used for trusts).

If using Nltest run:

Nltest /sc_verify:<DomainName>
You should get "Trust Verification Status = 0 0x0 NERR_Success" if not you must reset the secure channel.

Note:
This command doesn't work on a PDCe, so please don't reset the PDCe secure channel once you run it thus getting an error message.
Don't use /sc_query to verify secure channels because it doesn't verify the SC, it just tells you the information about the last established SC. (so you will get a valid answer even when you cannot reach a DC)

If using Netdom:

netdom verify /domain:<DomainName>

In order to verify Trusts:
(Trusts work in a similar way as Secure Channels, there is a TDO (Trust Domain Object) maintained in each trusting and trusted domain partition, which password has to be in sync, of not the trust gets broken).

netdom trust /Domain: /verify

Note:
Please take into account that depending on security settings between domains this command may fail, for more troubleshooting information see the article below:
Client, service, and program issues can occur if you change security settings and user rights assignments
https://support.microsoft.com/kb/823659

Hope it has been useful for you. Feel free to share your thoughts.