• If the deployment model is Classic, then add an endpoint on public port 3389 for RDP and on public port 22 for SSH. See the steps to add an endpoint here.

Unable to connect/RDP/SSH to the failed over virtual machine

Is Connect button grayed out on the virtual machine?

• If your Connect button is grayed out and you are not connected to Azure via an Express Route or Site-to-Site VPN connection then for ARM (Resource Manager deployment model) add a Public IP on the Network interface of the virtual machine as shown below. 
• If the deployment model is Classic, then add an endpoint on public port 3389 for RDP and on public port 22 for SSH. See the steps to add an endpoint here.

Is Connect button available (not grayed out) on the virtual machine?

• Check Boot diagnostics on your Virtual Machine and check for errors as listed in this article. Note: enabling any setting other than Boot Diagnostics would require Azure VM Agent to be installed in the virtual machine before the failover
• If the virtual machine has not started, try failing over to an older recovery point.
• If the application inside the virtual machine is not coming up, try failing over to an app-consistent recovery point.
• If the virtual machine is domain joined then ensure that domain controller is correctly functioning. You can do that by creating a new virtual machine in the same network and ensuring that it is able to join to the same domain on which the failed over virtual machine is expected to come up.
  • If the domain controller is not functioning correctly try logging in to the failed over virtual machine using a local administrator account
• If you are using a custom DNS server then ensure that it is reachable. You can do that by creating a new virtual in the same network and checking that it is able to do name resolution using the custom DNS Server

Recommended documents

• Basic RDP troubleshooting steps
• Advanced RPD troubleshooting steps
• How to Failover VM to Azure

Cannot find storage account to proceed

• Wait till discovery operation is complete.
• Check if the storage account  is registered with another vault.

Inquiry failed after registration and cannot find file-shares

• Please make sure that storage account registered with the vault exists (not been deleted after registering)
• Please make sure that the storage account is a normal storage account and not a premium storage account or ZRS storage account. You can open the storage account in portal and make sure it supports creation of file-shares. If not, then inquiry is expected to fail.

Cannot find file-share to protect

• Make sure it is not already protected in the same vault.

Configure protection fails

• Make sure the file-share exists (for more information, see job details for the error).
• Another not so common reason for you to encounter this issue would be, If you had tried configure-protection on a LOT (ex. >= 20) of file-shares at once, then please retry for the failed file-shares.

Unable to delete file-share even after disabling protection of the file-share

• It takes 48 hours for our service to clean-up all the snapshots taken while the file-share was protected. So please wait for 48 hours after deleting the file-share.

Restore of some of the files failed and the job “completed with warnings”

• The list of files for which restore failed will be available in the file-share. Its path can be found in restore job details.

Restore failed with “destination share is locked”

• You cannot have multiple restore operations to the same file-share running concurrently. One of them will fail. Please wait for the operation currently in progress and retry.

Backup is not getting triggered at same time after the day-light savings kicked in

• This is a known issue, and we are working on resolve it (as a workaround, you can modify protection with a new policy).

Identify servers with earlier versions of Azure backup agent

For Installations of Azure Backup agent and Azure backup server

• Step 1: Navigate to the Recovery Services Vault to which you have registered servers that are potentially being backed up by older versions of the agent. You can find a representative list of vaults with older Azure Backup agents in the Azure Backup Update alerts from Azure.
• Step 2: On the left-side Settings section of the Recovery Services Vault click on Backup Infrastructure under the Manage section
• Step 3: To check on Azure backup agents installed as part of Azure Backup server installations, go to Backup Management Servers under Management Servers. This will list the servers that have Azure Backup server installations along with the version number for the Azure Backup agent that the Azure Backup server installation is using. 
• Step 4: To check agent versions for Microsoft Azure Recovery Services (MARS) agent installations or the Azure Backup agent, go to Protected Servers under Management Servers and, click on Azure Backup agent under Backup Management Type. This will list the servers that have Azure Backup agent installations along with the version number for the installation. 
• Step 5: Sort the Azure Backup Agent version column by clicking on the Agent Version column for MARS Agent installations or the Azure Backup Agent Version column for Azure backup server installations
• Step 6: The above step will give you the list of servers with Azure Backup agents that have versions lower than 2.0.9083.0 or agent versions listed as blanks. These are the servers where Azure Backup agents needs to be updated.

Update the Azure Backup agent installation on the server

• Click on a row that has Azure Backup agent lower than 2.0.9083.0 or blank. This will open the server detail screen.

• Click on Connect to receive a remote desktop connection file to connect with the server or directly connect to the server via the Remote Desktop connection on your server.

NOTE If the listed server does not exist or has been decommissioned, you can ignore the remaining steps below and skip to the next server.

• Enter your administrative login details and login
• If your server or server’s proxy has limited internet access, ensure that firewall settings on the server/proxy are configured to allow the URL appropriate to the Azure cloud you are using

• Copy the Azure backup agent update installer to the server

• Run the installer. The Microsoft Azure Recovery Services Agent Upgrade wizard opens

• Click Next
• Click Upgrade

• The final confirmation screen indicates that Azure Backup agent has been successfully updated.

For customers of System Center Data Protection manager (SC DPM)

• Login to your SC DPM server as an Administrator
• Open the DPM Console
• Click on Management in the Lower left navigation of the console.
• Within the information that appears on the left navigation, look for the Azure Backup agent version information
• If the version is lower than 2.0.9083.0, download the latest Azure backup agent installer and run the installer on the DPM server to update the Azure Backup agent.

Need help? Contact support

Scheduled backup failed

• Follow these steps to troubleshoot and resolve the issue https://aka.ms/ScheduledBackupFailManualWorks
• Download and install the latest agent from https://aka.ms/azurebackup_agent
• Verify whether Windows Guest Agent service is running in services (services.msc). Try restart the Windows Guest Agent service and initiate the Backup.

The Microsoft Azure Recovery Service Agent was unable to connect to Microsoft Azure Backup

• Check network connectivity to Azure using steps listed in this article, fix issues if any and then reinitiated the backup jobs.
• If you have Antivirus software running on the same system, it might be blocking the backups from happening. Test by temporarily disabling the Anti-virus, reinitiate to see if backups are working fine. If you find this to be the reasons, then from your Antivirus product exclude the folders using steps listed in this article.

Microsoft Azure Backup failed to access the shadow copy because of inadequate disk space on the volume

• Ensure that the cache volume has free space corresponding to 5-10 percentage of the backup data size or at least 2.5GB, whichever is greater.
• If required, switch to a new cache location that meets the specified space requirements. Refer to this article for changing the cache location to a different volume with recommended disk size.
• Ensure scratch folder is not located on OS volume.

VSS failing inside VM:

• Ensure latest Integration services is installed and running:  In addition, ensure  Integration Service version is latest by following steps listed in this article. Follow steps listed in this article to keep Integration services on latest version
• Ensure you are not using Dynamic Disk inside the VM: App consistent snapshot does not support Dynamic disk. Open Disk Management (diskmgmt.msc) and check if you have Volume with Dynamic Disk Type.

• Ensure you are not using iSCSI disk attached inside the VM: This is not currently supported.
• Ensure Backup Service is enabled:  Check the Hyper-V settings > Integration Services  to ensure Back Service is not disabled Ensure there are no conflicts with apps taking VSS snapshots. Check to ensure if you have multiple applications trying to take VSS snapshot at the same time resulting is conflict. For example a Backup application taking VSS snapshots when ASR is scheduled to take snapshot (as specified in your replication policy).

• Ensure VSS Services are running and healthy: Log in to the guest virtual machine and check if all the VSS writers are healthy.
  • Open command prompt as admin. Run the below commands:
    • Vssadmin list writers
    • Vssadmin list shadows
    • Vssadmin list providers
  • Check the output. If there are any writers are in failed state, check the application event log within the VM for errors related to VSS operation to understand why the writer is in failed state.
  • You can try restarting the service associated with the failed writer within the VM from Services.msc
    • Volume Shadow Copy
    • Azure Site Recovery VSS Provider
  • Wait for a couple of hours once this is done to see if App-consistent snapshots are being generated successfully.
  • As a last resort try rebooting the VM, this may resolve services that are in unresponsive state.
• Check if your VM is experiencing High Churn: The Daily Data Change Rate for VMs displayed on the portal represents cumulative data churn rate of all protected disks of a given VM., this data churn rate can increase or stay at high levels depending on how busy the VM or applications running in the VM.
  • The Average Source Disk Data Churn is 2 MB/s for standard storage for Azure Site Recovery (more information see).  In addition you can find storage scalability limits in this article.
  • Run Deployment planner and review recommendations (for network and storage).

VSS failing inside Hyper-V Host

Cause 

Resolution 

1. Download and install the latest MARS Agent version 
2. Ensure you have a non-OS Disk attached to your Azure VM. This needs to be a different disk from the Temp disk.  
3. Stop the Backup engine by executing the following command in an elevated command prompt: 

PS C:\> Net stop obengine  

1. Locate the Scratch folder. Typically, it is at C:\Program Files\Microsoft Azure Recovery Services Agent\Scratch 
2. Do not move the files. Instead, copy the scratch-space folder to the non-OS disk with sufficient space.  
3. Update the following registry entries with the path to the new cache space folder. 

Registry path 

Registry Key 

Value 

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Azure Backup\Config 

ScratchLocation 

New cache folder location 

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Azure Backup\Config\CloudBackupProvider 

ScratchLocation 

New cache folder location 

1. Restart the Backup engine by executing the following command in an elevated command prompt: 

PS C:\> Net start obengine 

• Ensure your configuration adheres to support matrix (supported/not-supported configurations) and prerequisites recommended for Hyper-v to Azure.
• Ensure Hyper-V host prerequisites are met before enabling replication. If you are using VMM, then ensure its prerequisites are also met.
• Ensure Hyper-V Virtual Machine Management service is up and running.
• Check the logs listed in this article to understand the root cause of the issue.
• Ensure WMI is enabled and accessible:
  • Basic WMI Testing
  • WMI Troubleshooting
  • Troubleshooting problems with WMI scripts and WMI Services
• Ensure  Integration Service version is latest:
  • Check if your version of Integration Services is latest by following steps listed in this article
  • Follow steps listed in this article to keep Integration services on latest version

Initial replication stuck (or) Replication not progressing

• Check if the replication is Paused
  • Connect to the on-premises Hyper-V manager console, select the virtual machine, and see the replication health.
  • If replication health is Critical, right-click the virtual machine > Replication > View Replication Health.
  • If replication is paused, then click Resume Replication.
• Check if the Azure Site Recovery services are running. (Re)Start any service, which is not running and check if the problem still exists.
  • Hyper-V to Azure: On the Hyper-V server:
    • Virtual Machine Management service
    • Microsoft Azure Recovery Services Agent
    • Microsoft Azure Site Recovery Service
  • Hyper-V with VMM to Azure:
    • On Hyper-V host, ensure "Virtual Machine Management service" and "Microsoft Azure Recovery Services Agent" is running
    • On VMM server, ensure  "System Center Virtual Machine Manager Service" is running
  • Ensure WMI Provider Host service is running on Hyper-V host.
• Check your network bandwidth availability
  • Check if there are bandwidth or throttling constrains in your configuration by following steps and recommendations listed in this article.
  • Run the profiler (steps) using Deployment Planner tool. Follow bandwidth recommendations, storage recommendations and aware of data churn limitations.
• Check connectivity between Hyper-V server to Azure: From Hyper V server, open the Task Manager (press Ctrl-Shift-Esc) -> Performance tab ->Open Resource Monitor-> Network tab. Check if cbengine.exe in ‘Processes with Network Activity’ is actively sending large volume (in Mbs) of data.
• Check if Hyper-V server can connect Azure Blob:   Select and check cbengine.exe to view the 'TCP Connections' to see if there is connectivity from Process server to Azure Storage blob URL
• High data churn on VM: 
  • Check if your VM is marked for resynchronization. One of the reasons this can happen, is if the HRL files reaches 50% of the available disk space. To investigate the source of churn follow steps listed in this article. If this turns out to be the issue, then provision more storage space for all VMs enabled for replication to Azure that are accounting for the HRL file growth.
  • Network bandwidth limitations can also impact replication. Follows steps listed under Check your network bandwidth availability section to ensure there is enough bandwidth to meet your requirements.
  • Ensure the replication is not paused. If the replication is suspended/paused, it does not stop writing the changes to the hrl file (which in turn can lead growth of HRL files reaching limit).
  • Choose the appropriate storage to match your churn scenario as listed in this article.

• Check for conflicts: Ensure there are no other application taking VSS snapshots (ex. Backup application) when ASR is scheduled to take snapshot (as specified in your replication policy). By default, Site Recovery takes an app-consistent snapshot every 4 hours. You can configure any value between 1 and 12 hours. This can be done by manually triggering a snapshot using steps listed below:
  • Open command prompt as admin
  • Navigate to agent install directory and run the below command
    • C:\Program Files (x86)\Microsoft Azure Site Recovery\agent>vacp.exe -systemlevel
  • If  you see any other application using the VSS service then there is a conflict.  Review the replication policy to adjust snapshot frequency to prevent overlap.
  • In addition, check for VSS related errors in C:\program files(x86)\Microsoft azure site recovery\Agent\application data\Applicationpolicylogs file.
• Check the health of VSS Service:
  • Open command prompt as admin. Run the below commands:
    • Vssadmin list writers
    • Vssadmin list shadows
    • Vssadmin list providers

• If the output does not return values, then restart the below services:
  • Microsoft Software Shadow Copy Provider
  • Volume shadow copy

• Check if your VM is experiencing High Churn: The Daily Data Change Rate for VMs displayed on the portal represents cumulative data churn rate of all protected disks of a given VM., this data churn rate can increase or stay at high levels depending on how busy the VM or applications running in the VM.
  • The Average Source Disk Data Churn is 2 MB/s for standard storage for Azure Site Recovery (more information see).  In addition you can find storage scalability limits in this article.
  • Run Deployment planner and review recommendations (for network and storage). Check if you need to adjust the bandwidth and throttling as listed in this article.
  • Ensure the sizing requirements for Configuration/Process Server are met.

Check the following on SOURCE MACHINE:

• Check connectivity between Source Server-to-Configuration/Process server. Check port 443 on Source machine for outbound connectivity to Config server and port 9443 on Source machine for outbound connectivity to Process Server. From Source machine, use telnet (instead of PING) to connect to a config Server and Process server IP address using port 443 and 9443.  If unable to connect, allow port 443 and 9443 on the Source machine and check if the problem still exits.
• Check the status of following Service (Services.msc).  Start if it is not running and check if the problem still exists.            

InMage Scout VX Agent - Sentinel/Outpost InMage Scout Application Service

Check the following on PROCESS SERVER

• Check if these services are up and running on Process server:   

InMage Scout VX Agent - Sentinel/Outpost Microsoft Azure Recovery Services Agent Microsoft Azure Site Recovery Service cxprocessserver tmansvc (Re)Start any service which is not running and check if the problem still exists. 

• Check connectivity between Process Server-to-Azure:  From Process Server machine, open the Task Manager (press Ctrl-Shift-Esc) -> Performance tab ->Open Resource Monitor-> Network tab.   Check if cbengine.exe in ‘Processes with Network Activity’ is actively sending large volume (in Mbs) of data.     

• Check if Process server can connect Azure Blob:   Select and check cbengine.exe to view the 'TCP Connections' to see if there is connectivity from Process server to Azure Storage blob URL

• Check if Throttle bandwidth is not constrained on Process server.  Increase the bandwidth (ex. greater than 20Mbs) and check if the problem still exists.

Refer link : Site recovery deployment planner get-throughput, Recommendations with desired RPO as input, Recommendations with available bandwidth as input

• Check if Process server is able to connect to Azure Public IP address using port 443: 
  • Open the latest CBEngineCurr.Log from %programfiles%\Microsoft Azure Recovery Services Agent\Temp and search for the string ':443'  and 'Connectivity issues'.

• • If you find issues in the log, then telnet to Azure Public IP address found in the CBEngineCurr.Log using port 443.  If you are unable to connect, then check if the access issue is due to firewall or Proxy as described in next step.
• Check if IP address-based firewall on Process server are not blocking access.  If you are using an IP address-based firewall rules on the server, then ensure all the azure IPs are added to firewall configuration.
• Check if URL-based firewall on Process server are not blocking access.  If you are using an URL based firewall rules on the server, then ensure URLs  specified in the prerequisites are added to firewall configuration.
• Check if Proxy Setting on Process server are not blocking access.  If you are using a Proxy Server, ensure the steps outlines in 'Configure Outgoing Network' listed in this article are followed.  Also ensure, the proxy server name is resolving by the DNS server.

Check the following on Azure Portal:

• Check for the Last Heartbeat time on the azure portal.  On the portal-> click on Recovery Service Vault-> Replicated Items-> VM-> Properties blade.  Ensure Latest Heart Beat for Source Server and Process Server are showing latest time.  Also ensure IP addess are accurate.