This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 61%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDS%3C/text%3E%3C/svg%3E)
I deployed an Azure Automate Hybrid worker with Azure Arc. Azure Arc was deployed successfully and the server showed up.
I was able to add the server to the hybrid runbook worker group. The extension was successfully installed and is visible in the Azure Arc portal.
But as soon as I try to run any runbook on the server the following eventlog is logged:
Exception in the JobRuntimeDataService SandboxHub implementation [accountId={75fb789b-3594-49b2-bd25-78a070826513}][sandboxId={ff659a0d-c4af-4292-8d43-60175d2f3cd2}][methodName=GetJobAction][exception=JobRuntimeData.Client.JobRuntimeDataServiceClientException: Bad Request {"Message":"Could not authenticate. Certificate is not attached to request."}
[JobRuntimeData.Common.TransientFaultHandling.HttpRequestWithStatusException: Bad Request {"Message":"Could not authenticate. Certificate is not attached to request."}]
at JobRuntimeData.Client.JwtTokenAuthorizationHeaderBuilder.GetAuthorizationHeader() in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\JwtTokenAuthorizationHeaderBuilder.cs:line 56
at JobRuntimeData.Client.JobRuntimeDataServiceClient.AddAuthentication(WebClient webClient) in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\JobRuntimeDataServiceClient.cs:line 465
at JobRuntimeData.Client.JobRuntimeDataServiceClient.Get[T](String suffixUri, IDictionary`2 queryParameters, IDictionary`2 headers) in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\JobRuntimeDataServiceClient.cs:line 353
at JobRuntimeData.Client.Operations.JobOperations.JobOperations.GetJobActions(Guid accountId, Guid sandboxId) in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\Operations\JobOperations\JobOperations.cs:line 62
at Orchestrator.Runtime.JobRuntimeDataProvider.JobRuntimeDataProvider.GetJobAction(Guid sandboxId) in X:\bt\1222199\repo\src\Shared\Orchestrator.Runtime\JobRuntimeDataProvider\JobRuntimeDataProvider.cs:line 70
at Orchestrator.Runtime.JobRuntimeDataProvider.JobRuntimeDataProviderExceptionHandler.GetJobAction(Guid sandboxId) in X:\bt\1222199\repo\src\Shared\Orchestrator.Runtime\JobRuntimeDataProvider\JobRuntimeDataProviderExceptionHandler.cs:line 71
inner exception -> JobRuntimeData.Common.TransientFaultHandling.HttpRequestWithStatusException: Bad Request {"Message":"Could not authenticate. Certificate is not attached to request."}
at JobRuntimeData.Client.ImdsClient.<InvokeRequest>d__10`1.MoveNext() in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\ImdsClient.cs:line 111
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at JobRuntimeData.Client.ImdsClient.<GetMsiTokenAsync>d__9`1.MoveNext() in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\ImdsClient.cs:line 0
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at JobRuntimeData.Client.JwtTokenAuthorizationHeaderBuilder.<>c.<GetAuthorizationHeader>b__12_0() in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\JwtTokenAuthorizationHeaderBuilder.cs:line 46
at JobRuntimeData.Common.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func) in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Common\TransientFaultHandling\RetryPolicy.cs:line 137
at JobRuntimeData.Client.JwtTokenAuthorizationHeaderBuilder.GetAuthorizationHeader() in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\JwtTokenAuthorizationHeaderBuilder.cs:line 46
]
I am not using any Run-As credentials so there is no certificate involved there.
Someone has any idea what is missing now?
Hi @Döserich Gérald ,
As you have mentioned that you are not using Run As credential, so the hybrid extension must have been installed using the local system account but not using any credential asset. However, as per the error message, the issue is related to authentication due to certificate! So, as a first step to try diagnose and troubleshoot issues with extension-based Hybrid Runbook Workers, please check items mentioned in this Azure document and see if helps.
On the other hand, I am reaching out to our internal product team to understand any cause of the error and/or if there is any current limitation or issue with this extension-based Hybrid Runbook Workers. Will keep you updated as I hear more information.
Thanks for your response. Unfortunately this does not help me further.
I already ran the troubleshooter as well as the log extractor. The log file shows (the event viewer export) shows the same error.
There are not other errors in the Extension log. There is no worker / register / startup log in the extension. Only logs of the extension installation.
The 0.settings file shows the same url as the "RegistrationUrl" in the automation account. So I assume it is connected properly. It also shows the Server as online in Azure Arc as well as the connector checks-in in Azure Automate. The error only occurs if you try to run any runbook.
Everything is running on a fully patched Windows Server 2019 which is well in the required specs (4 Cores, 8 GB RAM)
Hi @Döserich Gérald ,
Please find the below update:
This is a known issue when MSI enabled Automation account use hybrid worker extension. The fix is currently under testing and ETA for the fix would be 13/12/2021. Until then, to mitigate this issue please disable MSI on Automation account for time being.
Disabling the System Assigned Managed Identity on the Automation Account does indeed seem to fix the issue for now.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 25%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJF%3C/text%3E%3C/svg%3E)
Any word here? I'm running into this issue myself.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 80%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETH%3C/text%3E%3C/svg%3E)
Same issue here though I received a different error
An unhandled exception has reached the sandbox's main entry point. The sandbox will exit immediately. [sandboxId={1d140299-09b9-4843-ab58-623ff3075b5a}][exceptionMessage=JobRuntimeData.Client.JobRuntimeDataServiceClientException: Unauthorized {"Message":""} ---> JobRuntimeData.Common.TransientFaultHandling.HttpRequestWithStatusException: Unauthorized {"Message":""}
at JobRuntimeData.Client.ImdsClient.<InvokeRequest>d__10`1.MoveNext() in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\ImdsClient.cs:line 111
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at JobRuntimeData.Client.ImdsClient.<GetMsiTokenAsync>d__9`1.MoveNext() in X:\bt\1222199\repo\src\Shared\JobRuntimeData.Client\ImdsClient.cs:line 0
--- End of stack trace from previous location where exception was thrown ---
I need the MSI so I'm not able to disable that functionality. For now I'll go oldskool with an agent based hybrid worker but that puts a hold on few things. We want to enable ARC on all on-premise servers for a certain functionality but now we can't for another functionality.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 85%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKR%3C/text%3E%3C/svg%3E)
Also hit by this issue, any updated news on fix date?
Hi @Döserich Gérald / @Joshua Frias / @Thomas Hofkens - Savaco / @Kenneth René Nielsen ,
Apologies for the delay here. The fix has been deployed to all regions. Please upgrade or update to latest version i.e., there are 2 ways to update:
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 34%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERC%3C/text%3E%3C/svg%3E)
This is the message I'm receiving when I go to the VM Extensions page in two different tenants: Extension updates are currently unavailable. Come back later for an improved experience.
Does this update apply to Linux Arc-enabled servers as well? Is removing and re-adding the server to Arc the best bet here?
Hi @Robbie Crash ,
The fix should work for Windows Arc VMs environment. Are you using Linux / Windows environment? And I assume that you have tried for Arc VMs but not for Azure VMs. Because for Arc VMs, we can upgrade without any issues.