Windows VMs won't start because the GuestAgent extension can't be found

Sergey Borovoy 501

Hi, I have a bunch of Windows Server 2019 VMs and they were working fine just yesterday, but this morning none of them can be started. Provisioning state becomes "failed" and the error message is "No version found in the artifact repository that satisfies the requested version '' for VM extension with publisher 'Microsoft.WindowsAzure.GuestAgent' and type 'CRPProd'"

So I called a colleague who also has a bunch of similar Windows Server VMs in a different subscription (created independently, those are not copies of my VMs) and he has the same problem.

Looks like some kind of Azure outage, GuestAgent extension became inaccessible? On the other had, I'm worried about the fact that the requested version is shown as an empty string. If this is what my VM requests for some reason, no wonder it can't find it.

The region is Western Europe, in case it matters.

Tried googling the error message, nothing really useful.

I'll appreciate your help.

Sushant Potdar 21 Reputation points

2021-10-13T07:44:27.223+00:00

Facing same issue since last 3-4 hours.
Sushant Potdar 21 Reputation points

2021-10-13T07:44:41.463+00:00

Facing same issue since last 3-4 hours.
Mark N | Sterling 6 Reputation points

2021-10-13T07:53:22.11+00:00

Same here, all our VM's are in a failed state this morning. West Europe region (UK)
No idea how to resolve
Sumit Waghmare 11 Reputation points

2021-10-13T07:53:43.577+00:00

Wow! same issue, I was troubleshooting and trying to create new VM
Central India
Waldo Nel 51 Reputation points

2021-10-13T07:55:40.777+00:00

Exact same issue in South Africa North. Been like this for about 2 hours now
Terho Antila 46 Reputation points

2021-10-13T08:02:19.59+00:00

I have the same problem. Tweeted and mentioned @AzureSupport. https://twitter.com/terhoantila/status/1448193642947137536
Rohan Raj Kumar Saini 11 Reputation points

2021-10-13T08:02:57.233+00:00

Same here in India
Mike Honey 116 Reputation points

2021-10-13T08:04:50.703+00:00

Azure Support just acknowledged this issue on twitter
https://twitter.com/AzureSupport/status/1448195809728675840?s=20

Same here - US West 2
Marius van Deventer 6 Reputation points

2021-10-13T08:08:54.67+00:00

Same, logged a MS Ticket. Super annoying.
Harsha Amarasiri 6 Reputation points

2021-10-13T08:09:56.163+00:00

Facing the exact same issue.
Jan Mahmud 1 Reputation point

2021-10-13T08:15:10.75+00:00

Same issue in UK South and UK West
Piet Baarssen 6 Reputation points

2021-10-13T08:21:06.613+00:00

Same problem here WEU region
VGP 11 Reputation points

2021-10-13T08:22:53.667+00:00

Have the same problem. We've multiple VMs in East US region. But one works one doesn't. Is this VM specific, can anyone let us know.
VGP 11 Reputation points

2021-10-13T08:23:05.957+00:00

Have the same problem. We've multiple VMs in East US region. But one works one doesn't. Is this VM specific, can anyone let us know.
Ricardo Salvador 11 Reputation points

2021-10-13T08:24:25.24+00:00

Same problem here! (West Europe)
Ricardo Salvador 11 Reputation points

2021-10-13T08:24:36.773+00:00

Same problem here! (West Europe)
Stanev, Vladimir (DFSaaS) 16 Reputation points

2021-10-13T08:25:21.173+00:00

Absolutely the same. The machines cannot be turned on, once they are turned off. Do not turn off the machines, otherwise you'll lose them for now.
Anna Moser 6 Reputation points

2021-10-13T08:25:35.287+00:00

Same issue for
MicrosoftWindowsServer
2019-datacenter-gensecond, datacenter-core-20h2-with-containers-smalldisk, 2022-datacenter-core-smalldisk-g2
deployments on germany west central

all our custom images are also affected.
Abhijit DebRoy 6 Reputation points

2021-10-13T08:28:20.857+00:00

It's been 1 hour and we are getting the same error, East US location. VMs could not be restarted or redeployed.
Gary Neilson 6 Reputation points

2021-10-13T08:42:22+00:00

Same on the official status page: https://status.azure.com/en-gb/status
Naganand Kanagal 6 Reputation points

2021-10-13T08:49:48.043+00:00

I have the same issue, I tried to restart a VM that I had created yesterday in the SEAsia region. Glad I found this thread.
Olivier Vanekem 81 Reputation points

2021-10-13T08:57:14.363+00:00

As many we also are impacted for regions West Europe and East US.
Waiting for the fix... (and considering multi-cloud redundancy in the future...)
Bryon Brill 11 Reputation points

2021-10-13T08:59:06.417+00:00

Someone Deployed some code, and it messed with the existing VM's. I'm certain they are in the process of rolling or reverting it back to previous state. It will take it time to replicate across regions.
Rodrigo Almeida 6 Reputation points

2021-10-13T09:04:30.123+00:00

West Europe here, only one of our 3 machines are affected, so far.
Kenn Thompson 16 Reputation points

2021-10-13T09:07:45.543+00:00

Same issue in Central US.
Kenn Thompson 16 Reputation points

2021-10-13T09:07:55.153+00:00

Same issue in Central US.
Graham Schofield 6 Reputation points

2021-10-13T09:08:29.78+00:00

Same here with my AVD estate, only got one server running at the moment and I cannot scale up for my user rush but with AVD you can at least increase the session limit on my single remaining server so my users are less grumpy at the moment.
Kenn Thompson 16 Reputation points

2021-10-13T09:17:08.94+00:00

Status page is woefully incorrect. North Central US also affected, shows green on status page.
John Nyabaro 6 Reputation points

2021-10-13T09:18:30.913+00:00

Facing the same issue here in Kenya in the last 4 hours
Lorenz Hänggi 6 Reputation points

2021-10-13T09:22:54.607+00:00

same issue here for VMSS VMs and SwitzerlandNorth

check out the status page
https://status.azure.com/en-us/status
George Norberg 6 Reputation points

2021-10-13T09:35:49.767+00:00

Same problem here with VM in North EU.
Raymond de Jong 76 Reputation points

2021-10-13T09:39:47.247+00:00

This issue might be the root cause that Azure DevOps is also having troubles.
Agents for Pipelines not starting

https://status.dev.azure.com/_event/266577720
Kiron Roy 11 Reputation points

2021-10-13T09:44:35.46+00:00

Same issue in US West region
Kiron Roy 11 Reputation points

2021-10-13T09:44:44.31+00:00

Same issue in US West region
Prasanna Venkateswaransachidanandam 6 Reputation points

2021-10-13T09:49:01.34+00:00

Having the same issue in US east. Please help.
Purna Rao 26 Reputation points

2021-10-13T09:50:40.877+00:00

Observing same issue in west europe region
Purna Rao 26 Reputation points

2021-10-13T10:00:55.867+00:00

Facing same issue in Central US region as well. Any ETA for the fix?
Damien Tacheron 66 Reputation points

2021-10-13T10:01:23.99+00:00

Since this morning, i have about 50 employees of a service provider that are unable to do their work.
We are an important energy supplier and our customer support center is completely stopped
And Azure considers this as a warning level problem, not even critical while this outage occurs sinc 4 hours
Vincenzo Sferruzza 41 Reputation points

2021-10-13T10:22:59.373+00:00

I agree. It's a global issue that should be classified as critical.
No one can work.
It's a shame!
Martin Kidd 6 Reputation points

2021-10-13T11:30:08.203+00:00

Looks like they are getting it resolved, I'm now able to start the VMs that were unable to start on schedule this morning. (Mine are all in the Canada East Datacentre)
JL 6 Reputation points

2021-10-13T11:51:02.973+00:00

My VM has restarted successfully right now!
Bryon Brill 11 Reputation points

2021-10-13T12:44:04.307+00:00

STOP the VM's and START the VM's having everyone pinging here isn't going to make it work any faster. clear the FAILED State by Stopping the VM. Then Starting it. I had to reboot a few of mine to get them to function properly.
Nora Ahmed 6 Reputation points

2022-06-02T23:05:26.477+00:00

×Close
comment previous version
There is a previous version saved. Would you like to load it or discard it
Amir Ahmed Salem Khalaf 326 Reputation points

2022-06-03T12:50:59.583+00:00

great
Sana'a Ma'rouf Al-Hussien (Al Sabaiba'a) 151 Reputation points

2022-06-03T13:47:02.09+00:00

Azure Support just acknowledged this issue on twitter
Javier Coreno 11 Reputation points

2022-06-13T22:35:33.367+00:00

I will try comment this post
Javier Coreno 11 Reputation points

2022-06-13T22:44:15.627+00:00

I will write this answer as a try.
Prakash Kunar 11 Reputation points

2022-10-14T08:06:20.39+00:00

the solution might be found soon
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Awan 5 Reputation points

2024-03-02T07:30:45.57+00:00

Its working for me almost month ago but now i am facing the exact same issue as well.
Nazar is real bro so Nazar ki dua also.
Hurajp 0 Reputation points

2024-05-01T17:03:43.3266667+00:00

hello Bro..

128 answers

David Whitham 21 Reputation points

2021-10-13T07:37:31.327+00:00

Hi exactly the same issue here - 11 servers offline with the same error, North Europe and UK South.
Please sign in to rate this answer.

4 people found this answer helpful.

0 comments No comments
Sign in to comment
Marius Sandbu 56 Reputation points MVP

2021-10-13T08:02:05.157+00:00

This issue affects all Windows Servers across all Azure regions. Azure CRP = Compute Resource Provider (Which is responsible for all VM actions by default uses the built-in extension WindowsAzureGuestAgent to communicate with the VM. Seems like Microsoft made some changes on the backend artifact so there is a version mismatch. As long as the version mismatch is there it is hard to make some changes.

Note that reregistering the resource provider for compute does not help and since this only affects the windows agents, linux servers are not affected. Or other VM's that does not use the Windows Guest Agent.
Please sign in to rate this answer.

4 people found this answer helpful.

0 comments No comments
Sign in to comment
Johan Groenewald 106 Reputation points

2021-10-13T08:25:00.28+00:00

Watching the health page and Twitter account is just infuriating at this point.

Health page says no issues. So you start redeploying VMs and getting the same error messages.

Then the response from Twitter : "Have you tried a different browser? Did you clear the cache?"

"No, we just felt like complaining without checking our side of Azure." Not sure why I have a certification from MS if I have to be told to try a different browser.
Please sign in to rate this answer.

4 people found this answer helpful.
Tiberiu Barna 16 Reputation points

2021-10-13T08:34:05.93+00:00

Waiting for "turning it off and back on" proposed solution...

Johan Groenewald 106 Reputation points

2021-10-13T08:39:46.37+00:00

If I received that question, would have responded with "yes". That's when the error occurs :)

Johan Groenewald 106 Reputation points

2021-10-13T09:45:39.83+00:00

3 hours later, health page now shows everything. That's not good that it takes them that long to update the health page.

So we can trust the health page by adding a 3hour delay
Sign in to comment
Pratik Soni 21 Reputation points

2021-10-13T11:27:47.84+00:00

Seems the issue has been resolved. VMs are now up and running in West US 2.
Please sign in to rate this answer.

4 people found this answer helpful.

0 comments No comments
Sign in to comment
Anonymous

2021-10-13T07:38:00.383+00:00

We have the same issue here how ever some VM's were started successfully just prior to 07:00 UK time.
Please sign in to rate this answer.

3 people found this answer helpful.

0 comments No comments
Sign in to comment