Cannot purge or recover deleted key vault

Chad Burggraf 6

I have a set of key vaults that are automatically created and destroyed as part of our infrastructure automation. Starting on August 8, 2021, one of these vaults is failing to purge after being deleted. All of the other vaults delete and purge just fine, and they are all identical except for naming and resource group membership.

Purge protection is not enabled for any of the vaults.

When purging via PowerShell (using Remove-AzKeyVault -InRemovedState -Force), the command hangs indefinitely. Our CI times out after 6 hours.

When purging via the portal UI, a success message is generated but the vault is never actually purged.

Attempting to recover the vault through the UI generates a conflict exception.

Any help resolving this issue is very much appreciated!

Marilee Turscak-MSFT 37,061 Reputation points Microsoft Employee

2021-08-11T21:11:22.017+00:00
Does the vault show up when you run Get-AzKeyVault -InRemovedState?

To permanently delete the vault, you can also try using:

az keyvault purge --name keyvaultname

https://learn.microsoft.com/en-us/cli/azure/keyvault?view=azure-cli-latest#az-keyvault-purge
Chad Burggraf 6 Reputation points

2021-08-11T22:42:53.747+00:00

Does the vault show up when you run Get-AzKeyVault -InRemovedState?

Yes, it does.

az keyvault purge --name keyvaultname

This command hangs indefinitely just like doing it from PowerShell.
Marilee Turscak-MSFT 37,061 Reputation points Microsoft Employee

2021-08-19T17:16:34.857+00:00

Are you logged in with a Global Admin account when you run the commands? This can happen if your account does not have sufficient privileges. I'll also note that if purge protection was ever enabled it cannot be disabled, and that could be a reason for this. I've also reached out to the Key Vault team for help on this.
Chad Burggraf 6 Reputation points

2021-08-19T17:23:00.833+00:00

Are you logged in with a Global Admin account when you run the commands?

Yes, I am.

I'll also note that if purge protection was ever enabled it cannot be disabled, and that could be a reason for this.

Purge protection was never enabled. The set of vaults involved in this operation are created and deleted (and purged) daily. This particular vault just started failing randomly, but the others continued to purge correctly.

I currently have the automation that creates and destroys this environment disabled because of this issue. I would love to get it resolved ASAP so that I can resume using this service.
Chad Burggraf 6 Reputation points

2021-09-09T12:34:34.943+00:00

Good morning, is there any update on this? This is now a blocker for us. Please note we have now passed the scheduled purge date and yet the vault persists:
David Norris-Hill 6 Reputation points

2021-10-22T14:15:50.7+00:00

@Marilee Turscak-MSFT Any update on this, we have 2 keyvaults that we cannot delete - the details are exactly the same as @Chad Burggraf described above?

1 answer

David Norris-Hill 6 Reputation points

2021-10-21T14:32:10.89+00:00

So we reported this issue to Microsoft who had to ask their engineering team to reset our vaults to deleted state which then allowed us to purge them finally.

They also said the following: "Delete and Purge operations don’t happen immediately; they usually take somewhere between 1-5m to complete, with a 2.5m rough average. Very occasionally, if you don't respect that and proceed to create > delete > purge in very quick succession, the key vault may get into a faulty state where our SLM jobs are not able to process the request immediately and the request becomes “stuck” and stays that way until released by engineering. That being said, this does happen very sporadically. To ensure that this doesn’t happen, an ideal workflow would be similar to the following: create KV > delete KV > wait 2.5m > purge KV."
Please sign in to rate this answer.

1 person found this answer helpful.
Richard 6 Reputation points

2024-06-27T12:52:39.1666667+00:00

I've hit this problem too - and this sounds like a bug in Azure keyvault resource management at Microsoft's end (please fix it!!). If a purge is not possible for 2.5m (or however long) the purge request (remove-azkeyvault -InRemovedState ... ) should fail in a timely manner with feedback (nothing for hours) and should be rerunnable successfully after 2.5m (or however long).
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Cannot purge or recover deleted key vault

1 answer

Your answer