Hybrid Workers is not picking up the jobs - Workers are listed but not refreshing the last seen

Question

Hi,

I have hybrid worker set up with promise - until few days ago, everything was running smoothly but it's now stop to picking the jobs. The jobs are queued but get suspended. I have installed hybrid worker and it does appears in the list but does not get update - and jobs get suspended.

I have followed the trouble shooting https://learn.microsoft.com/en-us/azure/automation/troubleshoot/hybrid-runbook-worker. Service "HealthService" been stop and running fine but it does not update the "Hybrid Worker Group" in azure.

has there been something changed recently in hybrid-runbook-worker and what could be the cause.

Br

Ganni

Answer 1

@Parajuli , Thank you for the question. Before getting to troubleshooting this issue, please confirm the type of hybrid worker group - whether it is agent based (which depends on the HealthService) Or if it is an extension based. For more details, see Automation Hybrid Runbook Worker overview.

You can get this information, by going to the Azure Automation Account --> Hybrid Worker Group --> select the group name --> Hybrid Workers.

Examine the Platform and Last Seen time column here. The platform column shows the type of worker (extension or agent based) and "Last seen time" shows the last time when the worker connected to Azure automation account.

Based on this information, either of the following article should be used for troubleshooting:

1. Agent based worker troubleshooting
2. Extension based worker troubleshooting

Also, for the jobs getting suspended, is it always getting suspended? Can you please check the following for more details related to this issue

1. What is the associated message related to Job suspension?
2. Are all types of runbooks getting suspended? If you create a sample runbook to just print "Hello World", does it run in Azure Automation sandbox but does not run when Hybrid Runbook worker is selected?
3. In Azure Automation --> Activity logs, do you see any error or warning around the same time?

Hope this helps.

Answer 2

• Firstly, Check the Heartbeat of the Virtual machine using below Kusto Query.
• Go to LogAnalytics Workspace > Select your workspace > Click on "Logs". Run below query.

Heartbeat
| where Computer == “ServerName.xxxxxxx.Local”

• Flush health service inside the server

**#Flush health service

Stop-Service HealthService -Force** **
Rename-Item "C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State" -NewName "C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State_old"**

Start-Service HealthService

• Remove HybridWorker

**#Remove HybridWorker
Remove-Item -Path "HKLM:\software\microsoft\hybridrunbookworker" -Recurse -Force

Restart-service healthservice**

• Check the Error Logs in "Log Name:      Operations Manager"
• Then, spot the Hybrid Runbook Worker Agent Rule by Searching "Microsoft.EnterpriseManagement.HealthService.AzureAutomation.HybridAgent" in the Operations Manager event.
• You can also Re-Install the OMS Extension. This might be helpful in your scenario where Last Seen is not updating on Azure portal.

---

(If the reply was helpful please don't forget to upvote or accept as answer, thank you)

Best regards,

Nitesh Pal