Azure backup not working after doing in-place upgrade from Windows 2012 to Windows 2016

Question

Azure backup not working after doing in-place upgrade from Windows 2012 to Windows 2016

Henrik Flodén 20

We have done in-place upgrade of a few servers in Azure from Windows Server 2012 to Windows Server 2016 (following these instructions https://learn.microsoft.com/en-us/azure/virtual-machines/windows-in-place-upgrade). It all worked surprisingly well but unfortunately now the daily backups are failing with Error Code GuestAgentPluginProcessingError. I have followed all the trouble shooting suggestions and have re-installed the Azure VM-agent without any luck.

Digging deep I have found errors being logged under C:\Packages\Plugins\Microsoft.Azure.RecoveryServices.VMSnapshot\1.0.82.0\Status. I here see the error code FailedBackupComponentInit.

I am guessing that a re-installation of the VMSnapshot agent is needed. Does anyone have suggestions on how to do that?

SadiqhAhmed-MSFT 49,326 Reputation points Microsoft Employee Moderator

2023-08-25T14:29:26.4+00:00
Hello @Henrik Flodén Thank you for contacting us and sorry for the inconvenience it may have caused!

I understand that you performed an inplace upgrade from Windows 2012 to Windows 2016 of your VM protected with Azure backup.

The error code GuestAgentPluginProcessingError indicates that there is an issue with the Azure VM Guest Agent. This error can occur if the Guest Agent is not up-to-date or if there is an issue with the extension.

To resolve this issue, please follow the steps below:

Ensure that the Azure VM Guest Agent is up-to-date. You can check the version of the Guest Agent by navigating to the VM in the Azure portal, selecting "Extensions" from the left-hand menu, and then selecting "Azure VM Guest Agent". If the Guest Agent is not up-to-date, you can update it by selecting "Update" from the "Azure VM Guest Agent" blade.

If the Guest Agent is up-to-date, try uninstalling and then reinstalling the Azure Backup extension. To do this, navigate to the VM in the Azure portal, select "Extensions" from the left-hand menu, and then select "Azure Backup". From the "Azure Backup" blade, select "Uninstall" and then "Install" to reinstall the extension.

If the issue persists, you can try resetting the Guest Agent. To do this, navigate to the VM in the Azure portal, select "Extensions" from the left-hand menu, and then select "Azure VM Guest Agent". From the "Azure VM Guest Agent" blade, select "Reset".

If none of these steps resolve the issue, please let me know and provide me with more details about the error message you are seeing.
SadiqhAhmed-MSFT 49,326 Reputation points Microsoft Employee Moderator

2023-08-28T06:04:02.98+00:00

@Henrik Flodén Have you had a chance to see the previous response? If the suggestions were helpful, click “Accept Answer” and Up-Vote. Feel free to reach out to us if you've additional questions in this regard.
Henrik Flodén 20 Reputation points

2023-08-28T13:27:00.9733333+00:00

Hi

When I look at the extensions for the VM in the Azure portal I do not see any extensions called "Azure VM Guest Agent" or "Azure Backup". What I see are these 4 extensions:
SadiqhAhmed-MSFT 49,326 Reputation points Microsoft Employee Moderator

2023-08-28T16:31:28.73+00:00

@Henrik Flodén The status shows "Provisioning succeeded" so it looks good. Can try these steps - https://learn.microsoft.com/en-us/troubleshoot/azure/virtual-machines/windows-azure-guest-agent#solution-2-manually-uninstall-and-reinstall-the-azure-vm-agent and let us know the results?
Henrik Flodén 20 Reputation points

2023-08-28T16:35:03.12+00:00

Hi

I have already done that without any change - I still get the same error.
SadiqhAhmed-MSFT 49,326 Reputation points Microsoft Employee Moderator

2023-08-28T17:06:52.1633333+00:00

Please send us an email at AzCommunity @ Microsoft dot com with subject: Attn: Sadiqh Ahmed with your subscription ID and a link to this forum thread for context.
Tito Iglesias 0 Reputation points

2023-09-06T07:17:33.8933333+00:00
Hello together

We have the same problem. Since the in-place upgrade from Windows Server 2012 to Windows Server 2016, we get the error message "GuestAgentPluginProcessingError" and "VM agent failed in processing Extension command." when trying to backup the Azure VM.

**
The following has already been done:

Manually uninstall and reinstall the Azure VM Agent:

1.Open Control Panel > Programs and Features and uninstall Azure VM Agent.

2.Start Task Manager and stop the following services..

3.Azure VM Agent service

RdAgent service

Windows Azure Telemetry Service

Windows Azure Network Agent service

Create a folder named OLD under C:\WindowsAzure.

4.Move all folders named Packages or GuestAgent to the OLD folder. Also, move all GuestAgent folders in C:\WindowsAzure\logs that start as GuestAgent_x.x.xxxxx to the OLD folder.

5.Download and install the latest version of the Windows Installer (MSI) agent. You must have administrator rights to complete the installation.

6.Install the Guest Agent by running the following msiexec command:

msiexec.exe /i c:\VMAgentMSI\WindowsAzureVmAgent.2.7.<version>.fre.msi /quiet /L*v c:\VMAgentMSI\msiexec.log.

7.check that the RdAgent, Azure VM Agent and Windows Azure Telemetry services are now running.

8.Check the WaAppAgent.log file to ensure that the latest version of Azure VM Agent is running. 9.

9.Delete the OLD folder under C:\WindowsAzure.

Further information:

Once the servers are down, the backup can be run successfully.

Like @Henrik Flodén, no "Azure VM Guest Agent" or "Azure Backup" extension is visible under the Azure VM.

The problem still occurs after the manual reinstallation. Has anyone already been able to fix the problem?
Henrik Flodén 20 Reputation points

2023-09-06T07:24:07.46+00:00

Tito, I have a ticket at Microsoft for this but have not been provided with any solution yet. I will update if I get a solution.

Br Henrik
Tito Iglesias 0 Reputation points

2023-09-06T07:30:38.2066667+00:00

Hi Henrik

Thanks for the feedback. I'm looking forward to hearing what Microsoft has to say.

Best regards
Tito
William Fulks 85 Reputation points

2023-09-11T12:42:23.2666667+00:00

I am currently experiencing the same problem after doing an inplace upgrade from Server 2012 to 2016. Did you ever get a resolution?
Henrik Flodén 20 Reputation points

2023-09-11T13:29:39.1533333+00:00

@William Fulks not yet but my support case is still open. @SadiqhAhmed-MSFT could you maybe engage with William or Tito to see if you can find the root cause?

Accepted answer

0 additional answers

Your answer

SadiqhAhmed-MSFT 49,326 Reputation points Microsoft Employee Moderator

2023-08-25T14:29:26.4+00:00

Hello @Henrik Flodén Thank you for contacting us and sorry for the inconvenience it may have caused!

I understand that you performed an inplace upgrade from Windows 2012 to Windows 2016 of your VM protected with Azure backup.

The error code GuestAgentPluginProcessingError indicates that there is an issue with the Azure VM Guest Agent. This error can occur if the Guest Agent is not up-to-date or if there is an issue with the extension.

To resolve this issue, please follow the steps below:

Ensure that the Azure VM Guest Agent is up-to-date. You can check the version of the Guest Agent by navigating to the VM in the Azure portal, selecting "Extensions" from the left-hand menu, and then selecting "Azure VM Guest Agent". If the Guest Agent is not up-to-date, you can update it by selecting "Update" from the "Azure VM Guest Agent" blade.

If the Guest Agent is up-to-date, try uninstalling and then reinstalling the Azure Backup extension. To do this, navigate to the VM in the Azure portal, select "Extensions" from the left-hand menu, and then select "Azure Backup". From the "Azure Backup" blade, select "Uninstall" and then "Install" to reinstall the extension.

If the issue persists, you can try resetting the Guest Agent. To do this, navigate to the VM in the Azure portal, select "Extensions" from the left-hand menu, and then select "Azure VM Guest Agent". From the "Azure VM Guest Agent" blade, select "Reset".

If none of these steps resolve the issue, please let me know and provide me with more details about the error message you are seeing.
SadiqhAhmed-MSFT 49,326 Reputation points Microsoft Employee Moderator

2023-08-28T06:04:02.98+00:00

@Henrik Flodén Have you had a chance to see the previous response? If the suggestions were helpful, click “Accept Answer” and Up-Vote. Feel free to reach out to us if you've additional questions in this regard.
Henrik Flodén 20 Reputation points

2023-08-28T13:27:00.9733333+00:00

Hi

When I look at the extensions for the VM in the Azure portal I do not see any extensions called "Azure VM Guest Agent" or "Azure Backup". What I see are these 4 extensions:
SadiqhAhmed-MSFT 49,326 Reputation points Microsoft Employee Moderator

2023-08-28T16:31:28.73+00:00

@Henrik Flodén The status shows "Provisioning succeeded" so it looks good. Can try these steps - https://learn.microsoft.com/en-us/troubleshoot/azure/virtual-machines/windows-azure-guest-agent#solution-2-manually-uninstall-and-reinstall-the-azure-vm-agent and let us know the results?
Henrik Flodén 20 Reputation points

2023-08-28T16:35:03.12+00:00

Hi

I have already done that without any change - I still get the same error.
SadiqhAhmed-MSFT 49,326 Reputation points Microsoft Employee Moderator

2023-08-28T17:06:52.1633333+00:00

Please send us an email at AzCommunity @ Microsoft dot com with subject: Attn: Sadiqh Ahmed with your subscription ID and a link to this forum thread for context.
Tito Iglesias 0 Reputation points

2023-09-06T07:17:33.8933333+00:00

Hello together

We have the same problem. Since the in-place upgrade from Windows Server 2012 to Windows Server 2016, we get the error message "GuestAgentPluginProcessingError" and "VM agent failed in processing Extension command." when trying to backup the Azure VM.

**
The following has already been done:

Manually uninstall and reinstall the Azure VM Agent:

1.Open Control Panel > Programs and Features and uninstall Azure VM Agent.

2.Start Task Manager and stop the following services..

3.Azure VM Agent service

RdAgent service

Windows Azure Telemetry Service

Windows Azure Network Agent service

Create a folder named OLD under C:\WindowsAzure.

4.Move all folders named Packages or GuestAgent to the OLD folder. Also, move all GuestAgent folders in C:\WindowsAzure\logs that start as GuestAgent_x.x.xxxxx to the OLD folder.

5.Download and install the latest version of the Windows Installer (MSI) agent. You must have administrator rights to complete the installation.

6.Install the Guest Agent by running the following msiexec command:

msiexec.exe /i c:\VMAgentMSI\WindowsAzureVmAgent.2.7.<version>.fre.msi /quiet /L*v c:\VMAgentMSI\msiexec.log.

7.check that the RdAgent, Azure VM Agent and Windows Azure Telemetry services are now running.

8.Check the WaAppAgent.log file to ensure that the latest version of Azure VM Agent is running. 9.

9.Delete the OLD folder under C:\WindowsAzure.

Further information:

Once the servers are down, the backup can be run successfully.

Like @Henrik Flodén, no "Azure VM Guest Agent" or "Azure Backup" extension is visible under the Azure VM.

The problem still occurs after the manual reinstallation. Has anyone already been able to fix the problem?
Henrik Flodén 20 Reputation points

2023-09-06T07:24:07.46+00:00

Tito, I have a ticket at Microsoft for this but have not been provided with any solution yet. I will update if I get a solution.

Br Henrik
Tito Iglesias 0 Reputation points

2023-09-06T07:30:38.2066667+00:00

Hi Henrik

Thanks for the feedback. I'm looking forward to hearing what Microsoft has to say.

Best regards
Tito
William Fulks 85 Reputation points

2023-09-11T12:42:23.2666667+00:00

I am currently experiencing the same problem after doing an inplace upgrade from Server 2012 to 2016. Did you ever get a resolution?
Henrik Flodén 20 Reputation points

2023-09-11T13:29:39.1533333+00:00

@William Fulks not yet but my support case is still open. @SadiqhAhmed-MSFT could you maybe engage with William or Tito to see if you can find the root cause?

Answer 1

William Fulks 85

We got this resolved this morning with the help of an MS tech. Here's what we did:

Stop the Windows Azure Guest Agent service
Rename the folder C:\Packages\Plugins\Microsoft.Azure.RecoveryServices.VMSnapshot
Export (to backup) then delete the following registry folder - HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Azure\HandlerState\Microsoft.Azure.RecoveryServices.VMSnapshot_XXXX
Start the Windows Azure Guest Agent service back
Trigger a manual backup and it completed after less than 10 minutes

After restarting the service you can watch it recreate the folder under C:\Packages\Plugins\

SadiqhAhmed-MSFT 49,326 Reputation points Microsoft Employee Moderator

2023-09-12T16:13:20.2333333+00:00

@William Fulks Thanks for sharing the solution with the community. This would certainly be helpful.
William Fulks 85 Reputation points

2023-09-12T17:30:55.23+00:00

You're welcome.

Share via

Azure backup not working after doing in-place upgrade from Windows 2012 to Windows 2016

0 additional answers

Your answer