DFS replication issue

Question

Hi everyone,

I am troubleshooting a DFS Replication issue between an Azure-hosted server and two on-premises partners and would appreciate a second opinion on the remediation plan.

Environment:

• SRV-AZURE-01: Azure VM (Server 2019)

SRV-ONPREM-01 & SRV-ONPREM-02: On-premises partners

Topology: Multiple replication groups syncing between Azure and On-prem.

The Issue: Following a suspected dirty shutdown (likely related to backup/snapshot activity), replication has stopped for most (but not all) folders on the Azure member (SRV-AZURE-01).

Event Viewer: We are seeing Event ID 5002 with the following error codes:

Error 9036 (Paused for backup or restore)

  Error 9033 (Request cancelled by shutdown)
  
     Error 1723 (RPC server too busy)
     
     __WMI Status Check:__ I ran the following command to check the state of the replicated folders on `SRV-AZURE-01`: `Get-WmiObject -Namespace root\MicrosoftDFS -Class DfsrReplicatedFolderInfo | Select ReplicationGroupName, ReplicatedFolderName, State`
     
     __Results:__
     
        Two folders (Multimedia, Archive) are in __State 4 (Normal)__.
        
           All other folders (Infrastructure, Shared Services, etc.) are in __State 3 (AutoRecovery)__.
           
           My understanding is that __State 3__ indicates the database is offline/locked due to the unexpected shutdown (Event 2213 scenario) and requires manual intervention to resume.
           

Proposed Fix: I am planning to run the following WMI command on SRV-AZURE-01 to force the database back online:

wmic /namespace:\\root\microsoftdfs path dfsrmachineinfo call Restore

My Questions:

Is this wmic ... call Restore command safe to run while some folders on the same server are already working (State 4)? Will it impact the active folders?

Given the "RPC server too busy" errors, should I be concerned about immediate backlog flooding once the state changes to Normal?

Thanks in advance for your help!Hi everyone,

I am troubleshooting a DFS Replication issue between an Azure-hosted server and two on-premises partners and would appreciate a second opinion on the remediation plan.

Environment:

SRV-AZURE-01: Azure VM (Server 2019)

SRV-ONPREM-01 & SRV-ONPREM-02: On-premises partners

Topology: Multiple replication groups syncing between Azure and On-prem.

The Issue: Following a suspected dirty shutdown (likely related to backup/snapshot activity), replication has stopped for most (but not all) folders on the Azure member (SRV-AZURE-01).

Event Viewer: We are seeing Event ID 5002 with the following error codes:

Error 9036 (Paused for backup or restore)

  Error 9033 (Request cancelled by shutdown)
  
     Error 1723 (RPC server too busy)
     
     __WMI Status Check:__ I ran the following command to check the state of the replicated folders on `SRV-AZURE-01`: `Get-WmiObject -Namespace root\MicrosoftDFS -Class DfsrReplicatedFolderInfo | Select ReplicationGroupName, ReplicatedFolderName, State`
     
     __Results:__
     
        Two folders (Multimedia, Archive) are in __State 4 (Normal)__.
        
           All other folders (Infrastructure, Shared Services, etc.) are in __State 3 (AutoRecovery)__.
           
           My understanding is that __State 3__ indicates the database is offline/locked due to the unexpected shutdown (Event 2213 scenario) and requires manual intervention to resume.
           

Proposed Fix: I am planning to run the following WMI command on SRV-AZURE-01 to force the database back online:

wmic /namespace:\\root\microsoftdfs path dfsrmachineinfo call Restore

My Questions:

Is this wmic ... call Restore command safe to run while some folders on the same server are already working (State 4)? Will it impact the active folders?

Given the "RPC server too busy" errors, should I be concerned about immediate backlog flooding once the state changes to Normal?

Thanks in advance for your help!

Answer 1

Running the wmic /namespace:\\root\microsoftdfs path dfsrmachineinfo call Restore command is intended to bring the DFS Replication database back online after a dirty shutdown. However, it is important to note that executing this command while some folders are already in State 4 (Normal) may not directly impact those active folders. The command is designed to address the offline/locked state of folders in State 3 (AutoRecovery).

That said, there are a few considerations:

1. Impact on Active Folders: While the command should not disrupt the folders in State 4, there is always a risk when making changes to a replication environment. It's advisable to monitor the replication closely after running the command to ensure stability.
2. RPC Server Busy Errors: The "RPC server too busy" errors indicate that the server is under load, which could lead to delays in processing replication requests. Once the state changes to Normal, there may be a backlog of changes to replicate, especially if the server was busy during the downtime. It’s wise to prepare for potential performance issues or delays in replication as the system catches up.

In summary, while the command is generally safe to run, ensure that you have a backup and monitor the system closely for any issues following the execution of the command.