HybridWorker on ARC server jobs fail with .NET error

Question

Prerequisites on Windows server installed (.NET framework 4.8 installed and Powershell 5.1)
OS Name: Microsoft Windows Server 2019 Standard
OS Version: 10.0.17763 N/A Build 17763
Arc agent was installed successfully via registration script and registered into resource group.
Extension for Log analytics was installed successfully
In the automation account the ARC server was added to Hybrid workers group successfully - the Hybrid worker extension installs successfully on the windows server.

However, any scheduled job in the automation account goes to queued and then Suspended.
At the corresponding time on the agent the Application event log has the error message as follows

.NET Runtime version : 4.0.30319.0 - This application could not be started.

output of DotnetVersions:
Microsoft .NET Framework 4 Client Profile
KB2468871
KB2468871v2
KB2478063
KB2533523
KB2544514
KB2600211
KB2600217
Microsoft .NET Framework 4 Extended
KB2468871
KB2468871v2
KB2478063
KB2533523
KB2544514
KB2600211
KB2600217

Why is there this error coming? Note that previously we had the V1 Hybrid worker working and it was all running OK.

Answer 1

Hi @Neil M

Welcome to Microsoft Q&A! Thanks for posting the question.

I understand that the scheduled runbooks are not running on your Arc-enabled Azure Automation Hybrid Worker and that you are getting the error below:

.NET Runtime version : 4.0.30319.0 - This application could not be started.  

Here are some of the questions that should help you narrow down the issue:

1. What is the exact error in the Application event logs? Does it give any additional information like the exact process/application for which the error is being reported?

2. Are non-scheduled runbooks failing as well? For example, if you try to run a sample runbook using the options and select the Hybrid Worker group, does it fail as well?

3. Are you sure that the runbook was scheduled to run on this particular server? The Hybrid worker group might contain other servers and the runbook could get scheduled to run there as well.

4. Since the error states that a .NET application could not be started, it could also mean that the installed .NET Framework runtime is corrupt. Are you seeing similar errors for other applications as well? You may try to launch "mmc", "eventvwr", "Server Manager", "services.msc" etc., and see if they fail to run too. If you observe similar errors with these applications too, you may use the .NET Framework Repair Tool to repair the installed version of .NET Framework runtime.

---
Apart from the probing questions above, the following troubleshooting guidelines should help you to narrow down the issue further:

a. Check the Application logs available under "Windows Logs" in Event Viewer on the Hybrid worker machine. This would contain logs related to installation failures/crashes happening for related processes.

b. Check the Operational logs available under "Applications and Services Logs --> Microsoft-SMA" in Event Viewer on the Hybrid worker machine. This would contain logs related to the functioning of runbook workers as well as execution of runbook jobs.

c. The Operations Manager logs under the "Applications and Services Logs" would contain errors related to connection to Log Analytics workspace.

d. Based on the question, I understand that you are using Extension-based Hybrid Runbook Worker (as you mentioned that you did not face such issue in v1). Please refer to troubleshooting guidelines available here, for checklist and available logs.

Please let me know if you have any questions.

---
Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.

Answer 2

Issue was fixed with http://blog.cloudspecialisterne.dk/net-error-1024-when-using-hybrid-worker/