Microsoft Azure Backup Server is failing online recovery point jobs for all servers. Online repository appears to have lost its registration. During attempts to reregister, getting encryption passphrase provided is incorrect

Question

Microsoft Azure Backup Server is failing online recovery point jobs for all servers. Online repository appears to have lost its registration. During attempts to reregister, getting encryption passphrase provided is incorrect

Steve Gerbec 20

I have an on prem Microsoft Azure Backup Server that is backing up 5 servers. All backups are succeeding, however all Online recovery points are failing with ID 100010.

When looking at Management - Online, there is one Online subscription: Azure Backup. It has the yellow triangle with a black !. When i click Configure, it opens up a wizard. I go through the wizard and get to the encryption settings. At that point, i click Generate passphrase and click finish. it asks me for my security pin, i get that from azure and enter it. Then it errors out saying encryption passphrase provided is incorrect. I found an old documented version of the passphrase, i tried inserting that instead of using generate and i get the same error message.

Is there a way that i can obtain the encryption key? If not, what steps do i have to take to configure a new passphrase?

SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-11-20T18:57:51.7233333+00:00

Hello @Steve Gerbec Thank you for contacting us on Microsoft Q&A platform and sorry for the inconvenience caused to you!

I understand that you are using Microsoft Azure Backup Server that is backing up 5 servers and receiving errors as "Online recovery points are failing with ID 100010."

Could you please follow the instructions from the troubleshooting guide here - https://learn.microsoft.com/en-us/azure/backup/backup-azure-scdpm-troubleshooting and let us know how it goes.
Steve Gerbec 20 Reputation points

2023-11-21T20:10:28.9733333+00:00

How do i tell which version of server i'm running?

The local recovery points succeed without an issue

The issue here is that I do not have access to the Azure RSV - thats what i'm trying to re-register, but am having issues with.
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-11-22T12:26:29.2466667+00:00

Hello @Anonymous , Thank you for contacting us on Microsoft Q&A platform and sorry for the inconvenience caused to you!

I understand that you are getting online backup failure with error ID 100010, it seems you have lost connectivity to the Recovery Service Vault which could only be reconfigured with the Passphrase you registered the server.

Please let me know what is the exact error which you are getting after clicking the configure option, using the passphrase which you have and register the MABS server.

If possible, please also share the CBengine logs so that I could have a look into it. The location of the logs would be found at 'C:\Program Files\Microsoft Azure Backup Server\DPM\MARS\Microsoft Azure Recovery Services Agent\Temp'.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-11-24T13:25:40.6133333+00:00

@Steve Gerbec Were you able to send the logs to Amardeep?
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-11-28T07:13:28.05+00:00

@Steve Gerbec We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.
Steve Gerbec 20 Reputation points

2023-11-28T20:58:27.7566667+00:00

Ah so i need to make sure that i have the same encryption key as was originally set up in order for this to work properly?

Is there any way that i can set up a new key? or will i need a new RSV for that?
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-11-29T07:56:41.6166667+00:00

Hello @Anonymous , you are correct the only way to register again to the same vault is to use the same passphrase which you used to register the server on the Vault.

There is a way to change the passphrase from Configure Option on Management tab but that is possible when the MABS Server is still registered on the vault and online backup and restore are working fine. In your case it is not possible as the MABS server is not registered to vault and online backups are failing.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-11-30T15:05:32.91+00:00

@Steve Gerbec Did the previous post answered your question? If yes, please do "Accept Answer" and up-vote it
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-01T19:20:28.4366667+00:00

@Steve Gerbec We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.
Steve Gerbec 20 Reputation points

2023-12-01T19:24:47.7233333+00:00

I was about to set up a new RSV, but logged into the existing one and found that we actually do have backups that are being sent to the existing one. I'm not sure how this can be failing and succeeding at the same time
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-12-04T07:55:19.16+00:00

Hello @Anonymous , you are correct the backups might be working on the previous RSV but now it is failing as you lost connectivity to the RSV. So, in order to start the backups in the existing RSV you need the same passphrase which you used to register the MABS server.
Steve Gerbec 20 Reputation points

2023-12-04T21:36:06.84+00:00

I just logged in to the Azure RSV, and it appears as though we are getting some backups after all in the RSV. I don't see the jobs as being successful in MABS, but I am seeing a "Latest recovery point" of today (12/4/23) in RSV under Backup Items. Although when i try to do a restore from MABS, it only shows Disk as available. Something seems awry here. Why is it that MABS thinks jobs are failing, but RSV thinks they are succeeding?

Additionally, when I look at the Management tab, under the Online section, I see Registration status as Registered, but when I look on the right hand side, it says Online subscriptions, and then below that a yellow triangle with an ! inside it next to where it says "Azure Backup". If I click on that, under details it says "Warning (Check the subscription alerts in Alerts view for details)". Where does one find this Alerts view?
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-05T10:31:13.0066667+00:00
@Steve Gerbec Could you please share the screenshots of what you noticed in the Azure portal?

For checking alerts, Open the Azure portal and navigate to the subscription that you want to view alerts for.

Click on Monitor in the left-hand menu or type in "Monitor" in the global search field

Click on Alerts.

You will list of alerts, try to find the one that relates to your issue.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-06T14:03:45.6666667+00:00

@Steve Gerbec Appreciate if you could share the screenshots as mentioned in the previous post.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-08T06:44:24.2066667+00:00

@Steve Gerbec We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.
Steve Gerbec 20 Reputation points

2023-12-08T16:02:34.6833333+00:00

I have 0 alerts in Monitor | Alerts

This is what i see from Recovery Services Vault | Backup Items

This is what i see from MABS under Management

And these are the online recovery point failed errors that we see every day. I know i had to cross those out for privacy, but they are the same servers as the above screenshot
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-12-11T08:23:40.3366667+00:00

Hello @Anonymous , thank you for sharing the screen shots with us. Request you to click on view details for any one of the backup items and share that screen shot with us as it seems the local recovery points information is getting updated for the backup item and not the online recovery point.
Steve Gerbec 20 Reputation points

2023-12-11T20:40:11.67+00:00

Here is the View details on one of the jobs
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-12-13T08:54:29.9033333+00:00

Hello @Anonymous , thank you for sharing the screen shot. As expected you can check it is giving you the update for local recovery points on MABS server and Cloud Recovery Point shows nothing. So this is a expected behaviour if online backups are getting failed on MABS server.

As suggested, you have to register your MABS server again with the same Passphrase which was initially used for getting your online backups to start working again on MABS server.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-18T05:37:53.4333333+00:00

@Steve Gerbec Any progress on this?
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-20T09:49:58.7033333+00:00

@Steve Gerbec We haven't heard back from you. In case you already found a solution, would you please share it here with the community? Otherwise, let us know and we will continue to engage with you on the issue.
Steve Gerbec 20 Reputation points

2023-12-20T21:30:28.32+00:00

I opened a ticket with Microsoft. We ended up uninstalling the MARS agent on the MAB server, rebooting and then reinstalling the MARS agent fresh. We did confirm that I did not have the proper encryption password for the RSV, so I created a new RSV and registered the MAB server to that. Then we ended up opening each protection group and just nexting through everything so that it would get the setting of the new online location saved. After that, the online backups have been working as expected.

Thank you for the help! We can consider this issue resolved.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-21T09:45:11.45+00:00

@Steve Gerbec Thanks for the update. Glad to hear it is now resolved.

I will re-post the solution answer to your original question. That way, you can "Accept Answer" and "Up-vote" so that the relevancy of the post will improve when anyone in the community search for a similar query.

Accepted answer

0 additional answers

Your answer

SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-11-20T18:57:51.7233333+00:00

Hello @Steve Gerbec Thank you for contacting us on Microsoft Q&A platform and sorry for the inconvenience caused to you!

I understand that you are using Microsoft Azure Backup Server that is backing up 5 servers and receiving errors as "Online recovery points are failing with ID 100010."

Could you please follow the instructions from the troubleshooting guide here - https://learn.microsoft.com/en-us/azure/backup/backup-azure-scdpm-troubleshooting and let us know how it goes.
Steve Gerbec 20 Reputation points

2023-11-21T20:10:28.9733333+00:00

How do i tell which version of server i'm running?

The local recovery points succeed without an issue

The issue here is that I do not have access to the Azure RSV - thats what i'm trying to re-register, but am having issues with.
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-11-22T12:26:29.2466667+00:00

Hello @Anonymous , Thank you for contacting us on Microsoft Q&A platform and sorry for the inconvenience caused to you!

I understand that you are getting online backup failure with error ID 100010, it seems you have lost connectivity to the Recovery Service Vault which could only be reconfigured with the Passphrase you registered the server.

Please let me know what is the exact error which you are getting after clicking the configure option, using the passphrase which you have and register the MABS server.

If possible, please also share the CBengine logs so that I could have a look into it. The location of the logs would be found at 'C:\Program Files\Microsoft Azure Backup Server\DPM\MARS\Microsoft Azure Recovery Services Agent\Temp'.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-11-24T13:25:40.6133333+00:00

@Steve Gerbec Were you able to send the logs to Amardeep?
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-11-28T07:13:28.05+00:00

@Steve Gerbec We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.
Steve Gerbec 20 Reputation points

2023-11-28T20:58:27.7566667+00:00

Ah so i need to make sure that i have the same encryption key as was originally set up in order for this to work properly?

Is there any way that i can set up a new key? or will i need a new RSV for that?
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-11-29T07:56:41.6166667+00:00

Hello @Anonymous , you are correct the only way to register again to the same vault is to use the same passphrase which you used to register the server on the Vault.

There is a way to change the passphrase from Configure Option on Management tab but that is possible when the MABS Server is still registered on the vault and online backup and restore are working fine. In your case it is not possible as the MABS server is not registered to vault and online backups are failing.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-11-30T15:05:32.91+00:00

@Steve Gerbec Did the previous post answered your question? If yes, please do "Accept Answer" and up-vote it
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-01T19:20:28.4366667+00:00

@Steve Gerbec We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.
Steve Gerbec 20 Reputation points

2023-12-01T19:24:47.7233333+00:00

I was about to set up a new RSV, but logged into the existing one and found that we actually do have backups that are being sent to the existing one. I'm not sure how this can be failing and succeeding at the same time
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-12-04T07:55:19.16+00:00

Hello @Anonymous , you are correct the backups might be working on the previous RSV but now it is failing as you lost connectivity to the RSV. So, in order to start the backups in the existing RSV you need the same passphrase which you used to register the MABS server.
Steve Gerbec 20 Reputation points

2023-12-04T21:36:06.84+00:00

I just logged in to the Azure RSV, and it appears as though we are getting some backups after all in the RSV. I don't see the jobs as being successful in MABS, but I am seeing a "Latest recovery point" of today (12/4/23) in RSV under Backup Items. Although when i try to do a restore from MABS, it only shows Disk as available. Something seems awry here. Why is it that MABS thinks jobs are failing, but RSV thinks they are succeeding?

Additionally, when I look at the Management tab, under the Online section, I see Registration status as Registered, but when I look on the right hand side, it says Online subscriptions, and then below that a yellow triangle with an ! inside it next to where it says "Azure Backup". If I click on that, under details it says "Warning (Check the subscription alerts in Alerts view for details)". Where does one find this Alerts view?
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-05T10:31:13.0066667+00:00

@Steve Gerbec Could you please share the screenshots of what you noticed in the Azure portal?

For checking alerts, Open the Azure portal and navigate to the subscription that you want to view alerts for.

Click on Monitor in the left-hand menu or type in "Monitor" in the global search field

Click on Alerts.

You will list of alerts, try to find the one that relates to your issue.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-06T14:03:45.6666667+00:00

@Steve Gerbec Appreciate if you could share the screenshots as mentioned in the previous post.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-08T06:44:24.2066667+00:00

@Steve Gerbec We haven't heard back from you. Please reply if you have any questions in this matter and we will gladly continue the discussion.
Steve Gerbec 20 Reputation points

2023-12-08T16:02:34.6833333+00:00

I have 0 alerts in Monitor | Alerts

This is what i see from Recovery Services Vault | Backup Items

This is what i see from MABS under Management

And these are the online recovery point failed errors that we see every day. I know i had to cross those out for privacy, but they are the same servers as the above screenshot
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-12-11T08:23:40.3366667+00:00

Hello @Anonymous , thank you for sharing the screen shots with us. Request you to click on view details for any one of the backup items and share that screen shot with us as it seems the local recovery points information is getting updated for the backup item and not the online recovery point.
Steve Gerbec 20 Reputation points

2023-12-11T20:40:11.67+00:00

Here is the View details on one of the jobs
Amardeep Saini 1,735 Reputation points Microsoft Employee

2023-12-13T08:54:29.9033333+00:00

Hello @Anonymous , thank you for sharing the screen shot. As expected you can check it is giving you the update for local recovery points on MABS server and Cloud Recovery Point shows nothing. So this is a expected behaviour if online backups are getting failed on MABS server.

As suggested, you have to register your MABS server again with the same Passphrase which was initially used for getting your online backups to start working again on MABS server.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-18T05:37:53.4333333+00:00

@Steve Gerbec Any progress on this?
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-20T09:49:58.7033333+00:00

@Steve Gerbec We haven't heard back from you. In case you already found a solution, would you please share it here with the community? Otherwise, let us know and we will continue to engage with you on the issue.
Steve Gerbec 20 Reputation points

2023-12-20T21:30:28.32+00:00

I opened a ticket with Microsoft. We ended up uninstalling the MARS agent on the MAB server, rebooting and then reinstalling the MARS agent fresh. We did confirm that I did not have the proper encryption password for the RSV, so I created a new RSV and registered the MAB server to that. Then we ended up opening each protection group and just nexting through everything so that it would get the setting of the new online location saved. After that, the online backups have been working as expected.

Thank you for the help! We can consider this issue resolved.
SadiqhAhmed-MSFT 49,331 Reputation points Microsoft Employee Moderator

2023-12-21T09:45:11.45+00:00

@Steve Gerbec Thanks for the update. Glad to hear it is now resolved.

I will re-post the solution answer to your original question. That way, you can "Accept Answer" and "Up-vote" so that the relevancy of the post will improve when anyone in the community search for a similar query.

Answer 1

Hello @Steve Gerbec Much appreciate your patience throughout the support tenure on Microsoft Q&A platform and with Azure technical support team.

"Sharing the summary of the issue for benefit of broader community"

Issue: Microsoft Azure Backup Server is failing online recovery point jobs for all servers.

Cause: The passphrase was missing/lost causing the online recovery points failures.

Resolution: Uninstalled MARS agent rebooted server and re-installed the agent.

Created a new Recovery Services Vault (RSV) and generated a new passphrase and registered the Microsoft Azure Backup Server (MABS) to the RSV.

Nesting through each protection group to ensure that it would get the setting of the new online location saved. Post these steps the online backups started working as expected.

Hope this helps others looking for solution to the similar issues.

@Steve Gerbec Thanks once again for your kind gestures.

Happy holidays!

If the response helped, do "Accept Answer" and up-vote it

Share via

Microsoft Azure Backup Server is failing online recovery point jobs for all servers. Online repository appears to have lost its registration. During attempts to reregister, getting encryption passphrase provided is incorrect

0 additional answers

Your answer