Troubleshoot Azure Arc-enabled servers agent connection issues

This article provides information for troubleshooting issues that may occur configuring the Azure Connected Machine agent for Windows or Linux. Both the interactive and at-scale installation methods when configuring connection to the service are included. For general information, see Azure Arc-enabled servers overview.

Agent error codes

Use the following table to identify and resolve issues when configuring the Azure Arc-enabled servers agent. You will need the AZCM0000 ("0000" can be any four digit number) error code printed to the console or script output.

Error code Probable cause Suggested remediation
AZCM0000 The action was successful N/A
AZCM0001 An unknown error occurred Contact Microsoft Support for assistance.
AZCM0011 The user canceled the action (CTRL+C) Retry the previous command.
AZCM0012 The access token is invalid If authenticating via access token, obtain a new token and try again. If authenticating via service principal or device logins, contact Microsoft Support for assistance.
AZCM0016 Missing a mandatory parameter Review the error message in the output to identify which parameters are missing. For the complete syntax of the command, run azcmagent <command> --help.
AZCM0018 The command was executed without administrative privileges Retry the command in an elevated user context (administrator/root).
AZCM0019 The path to the configuration file is incorrect Ensure the path to the configuration file is correct and try again.
AZCM0023 The value provided for a parameter (argument) is invalid Review the error message for more specific information. Refer to the syntax of the command (azcmagent <command> --help) for valid values or expected format for the arguments.
AZCM0026 There is an error in network configuration or some critical services are temporarily unavailable Check if the required endpoints are reachable (for example, hostnames are resolvable, endpoints are not blocked). If the network is configured for Private Link Scope, a Private Link Scope resource ID must be provided for onboarding using the --private-link-scope parameter.
AZCM0041 The credentials supplied are invalid For device logins, verify that the user account specified has access to the tenant and subscription where the server resource will be created1.
For service principal logins, check the client ID and secret for correctness, the expiration date of the secret2, and that the service principal is from the same tenant where the server resource will be created1.
1See How to find your Azure Active Directory tenant ID.
2In Azure portal, open Azure Active Directory and select the App registration blade. Select the application to be used and the Certificates and secrets within it. Check whether the expiration data has passed. If it has, create new credentials with sufficient roles and try again. See Connected Machine agent prerequisites-required permissions.
AZCM0042 Creation of the Azure Arc-enabled server resource failed Review the error message in the output to identify the cause of the failure to create resource and the suggested remediation. For permission issues, see Connected Machine agent prerequisites-required permissions for more information.
AZCM0043 Deletion of the Azure Arc-enabled server resource failed Verify that the user/service principal specified has permissions to delete Azure Arc-enabled server/resources in the specified group — see Connected Machine agent prerequisites-required permissions.
If the resource no longer exists in Azure, use the --force-local-only flag to proceed.
AZCM0044 A resource with the same name already exists Specify a different name for the --resource-name parameter or delete the existing Azure Arc-enabled server in Azure and try again.
AZCM0062 An error occurred while connecting the server Review the error message in the output for more specific information. If the error occurred after the Azure resource was created, delete this resource before retrying.
AZCM0063 An error occurred while disconnecting the server Review the error message in the output for more specific information. If this error persists, delete the resource in Azure, and then run azcmagent disconnect --force-local-only on the server.
AZCM0067 The machine is already connected to Azure Run azcmagent disconnect to remove the current connection, then try again.
AZCM0068 Subscription name was provided, and an error occurred while looking up the corresponding subscription GUID. Retry the command with the subscription GUID instead of subscription name.
AZCM0061
AZCM0064
AZCM0065
AZCM0066
AZCM0070
The agent service is not responding or unavailable Verify the command is run in an elevated user context (administrator/root). Ensure that the HIMDS service is running (start or restart HIMDS as needed) then try the command again.
AZCM0081 An error occurred while downloading the Azure Active Directory managed identity certificate If this message is encountered while attempting to connect the server to Azure, the agent won't be able to communicate with the Azure Arc service. Delete the resource in Azure and try connecting again.
AZCM0101 The command was not parsed successfully Run azcmagent <command> --help to review the command syntax.
AZCM0102 An error occurred while retrieving the computer hostname Retry the command and specify a resource name (with parameter --resource-name or –n). Use only alphanumeric characters, hyphens and/or underscores; note that resource name cannot end with a hyphen or underscore.
AZCM0103 An error occurred while generating RSA keys Contact Microsoft Support for assistance.
AZCM0105 An error occurred while downloading the Azure Active Directory managed identify certificate Delete the resource created in Azure and try again.
AZCM0147-
AZCM0152
An error occurred while installing Azcmagent on Windows Review the error message in the output for more specific information.
AZCM0127-
AZCM0146
An error occurred while installing Azcmagent on Linux Review the error message in the output for more specific information.

Agent verbose log

Before following the troubleshooting steps described later in this article, the minimum information you need is the verbose log. It contains the output of the azcmagent tool commands, when the verbose (-v) argument is used. The log files are written to %ProgramData%\AzureConnectedMachineAgent\Log\azcmagent.log for Windows, and Linux to /var/opt/azcmagent/log/azcmagent.log.

Windows

Following is an example of the command to enable verbose logging with the Connected Machine agent for Windows when performing an interactive installation.

& "$env:ProgramFiles\AzureConnectedMachineAgent\azcmagent.exe" connect --resource-group "resourceGroupName" --tenant-id "tenantID" --location "regionName" --subscription-id "subscriptionID" --verbose

Following is an example of the command to enable verbose logging with the Connected Machine agent for Windows when performing an at-scale installation using a service principal.

& "$env:ProgramFiles\AzureConnectedMachineAgent\azcmagent.exe" connect `
  --service-principal-id "{serviceprincipalAppID}" `
  --service-principal-secret "{serviceprincipalPassword}" `
  --resource-group "{ResourceGroupName}" `
  --tenant-id "{tenantID}" `
  --location "{resourceLocation}" `
  --subscription-id "{subscriptionID}"
  --verbose

Linux

Following is an example of the command to enable verbose logging with the Connected Machine agent for Linux when performing an interactive installation.

Note

You must have root access permissions on Linux machines to run azcmagent.

azcmagent connect --resource-group "resourceGroupName" --tenant-id "tenantID" --location "regionName" --subscription-id "subscriptionID" --verbose

Following is an example of the command to enable verbose logging with the Connected Machine agent for Linux when performing an at-scale installation using a service principal.

azcmagent connect \
  --service-principal-id "{serviceprincipalAppID}" \
  --service-principal-secret "{serviceprincipalPassword}" \
  --resource-group "{ResourceGroupName}" \
  --tenant-id "{tenantID}" \
  --location "{resourceLocation}" \
  --subscription-id "{subscriptionID}"
  --verbose

Agent connection issues to service

The following table lists some of the known errors and suggestions on how to troubleshoot and resolve them.

Message Error Probable cause Solution
Failed to acquire authorization token device flow Error occurred while sending request for Device Authorization Code: Post https://login.windows.net/fb84ce97-b875-4d12-b031-ef5e7edf9c8e/oauth2/devicecode?api-version=1.0: dial tcp 40.126.9.7:443: connect: network is unreachable. Cannot reach login.windows.net endpoint Verify connectivity to the endpoint.
Failed to acquire authorization token device flow Error occurred while sending request for Device Authorization Code: Post https://login.windows.net/fb84ce97-b875-4d12-b031-ef5e7edf9c8e/oauth2/devicecode?api-version=1.0: dial tcp 40.126.9.7:443: connect: network is Forbidden. Proxy or firewall is blocking access to login.windows.net endpoint. Verify connectivity to the endpoint and it is not blocked by a firewall or proxy server.
Failed to acquire authorization token device flow Error occurred while sending request for Device Authorization Code: Post https://login.windows.net/fb84ce97-b875-4d12-b031-ef5e7edf9c8e/oauth2/devicecode?api-version=1.0: dial tcp lookup login.windows.net: no such host. Group Policy Object Computer Configuration\ Administrative Templates\ System\ User Profiles\ Delete user profiles older than a specified number of days on system restart is enabled. Verify the GPO is enabled and targeting the affected machine. See footnote 1 for further details.
Failed to acquire authorization token from SPN Failed to execute the refresh request. Error = 'Post https://login.windows.net/fb84ce97-b875-4d12-b031-ef5e7edf9c8e/oauth2/token?api-version=1.0: Forbidden' Proxy or firewall is blocking access to login.windows.net endpoint. Verify connectivity to the endpoint and it is not blocked by a firewall or proxy server.
Failed to acquire authorization token from SPN Invalid client secret is provided Wrong or invalid service principal secret. Verify the service principal secret.
Failed to acquire authorization token from SPN Application with identifier 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' was not found in the directory 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant Incorrect service principal and/or Tenant ID. Verify the service principal and/or the tenant ID.
Get ARM Resource Response The client 'username@domain.com' with object id 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' does not have authorization to perform action 'Microsoft.HybridCompute/machines/read' over scope '/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourcegroups/myResourceGroup/providers/Microsoft.HybridCompute/machines/MSJC01' or the scope is invalid. If access was recently granted, please refresh your credentials."}}" Status Code=403 Wrong credentials and/or permissions Verify you or the service principal is a member of the Azure Connected Machine Onboarding role.
Failed to AzcmagentConnect ARM resource The subscription is not registered to use namespace 'Microsoft.HybridCompute' Azure resource providers are not registered. Register the resource providers.
Failed to AzcmagentConnect ARM resource Get https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourcegroups/myResourceGroup/providers/Microsoft.HybridCompute/machines/MSJC01?api-version=2019-03-18-preview: Forbidden Proxy server or firewall is blocking access to management.azure.com endpoint. Verify connectivity to the endpoint and it is not blocked by a firewall or proxy server.

1If this GPO is enabled and applies to machines with the Connected Machine agent, it deletes the user profile associated with the built-in account specified for the himds service. As a result, it also deletes the authentication certificate used to communicate with the service that is cached in the local certificate store for 30 days. Before the 30-day limit, an attempt is made to renew the certificate. To resolve this issue, follow the steps to disconnect the agent and then re-register it with the service running azcmagent connect.

Next steps

If you don't see your problem here or you can't resolve your issue, try one of the following channels for additional support:

  • Get answers from Azure experts through Microsoft Q&A.

  • Connect with @AzureSupport, the official Microsoft Azure account for improving customer experience. Azure Support connects the Azure community to answers, support, and experts.

  • File an Azure support incident. Go to the Azure support site, and select Get Support.