Troubleshooting Revocation Scenarios for your Customer-Managed Keys Configured Azure Cosmos DB account
APPLIES TO: NoSQL MongoDB Cassandra Gremlin Table
Data stored in your Azure Cosmos DB account is automatically and seamlessly encrypted with keys that the customer manages as a second layer of encryption. When the Azure Cosmos DB account can no longer access the Azure Key Vault key per the Azure Cosmos DB account setting (see KeyVaultKeyUri), the account goes into revoke state. In this state, the only operations allowed are account updates that refresh the current assigned default identity or account deletion. Data plane operations like reading or writing documents are restricted.
This troubleshooting guide shows you how to restore access when running into the most common errors with Customer managed keys. Check either the error message received each time a restricted operation is performed or by reading the customerManagedKeyStatus property on your Azure Cosmos DB account.
Default Identity is unauthorized to access the Azure Key Vault key
Reason for error
You see the error when the default identity associated with the Azure Cosmos DB account is no longer authorized to perform either a get, a wrap or unwrap call to the Key Vault or your key is disabled or expired.
Troubleshooting
Please verify that your key is neither disabled or expired. In the contrary, when using access policies, verify that the get, wrap, and unwrap permissions on your Key Vault are assigned to the identity set as the default identity for the respective Azure Cosmos DB account.
In case you're using RBAC, verify that the "Key Vault Crypto Service Encryption User" role to the default identity is assigned.
Another option is to create a new identity with the expected permission and set it as the new default identity via the Azure Cosmos DB account update operation.
After assigning the permissions, wait upwards to one hour for the account to stop being in revoke state. If the issue isn't resolved after more than two hours, contact customer service.
Microsoft Entra Token Acquisition error
Reason for error
You see this error when Azure Cosmos DB is unable to obtain the default's identity Microsoft Entra access token. The token is used for communicating with the Azure Key Vault in order to wrap and unwrap the data encryption key.
Troubleshooting
Make sure that the current default identity assigned to the Azure Cosmos DB account is that of an existing Azure resource with all the correspondent permissions to access the Azure Key Vault.
A troubleshooting solution, for example, would be to create a new identity with the expected permission and set it as the new default identity via the Azure Cosmos DB account update operation.
After updating the account's default identity, you need to wait upwards to one hour for the account to stop being in revoke state. If the issue isn't resolved after more than two hours, contact customer service.
Azure Key Vault Resource not found
Reason for error
You see this error when the Azure Key Vault or specified Key are not found.
Troubleshooting
Check if the Azure Key Vault or the specified key exist and restore them if accidentally got deleted, then wait for one hour. If the issue isn't resolved after more than 2 hours, contact customer service.
Azure key Disabled or expired
Reason for error
You see this error when the Azure Key Vault key has been expired or deleted.
Troubleshooting
If your key has been disabled please enable it. If it has been expired please un-expire it, and once the account is not revoked anymore feel free to rotate the key as Azure Cosmos DB will update the key version once the account is online.
Invalid Azure Cosmos DB default identity
Reason for error
The Azure Cosmos DB account goes into revoke state if it doesn't have any of these identity types set as a default identity:
- FirstPartyIdentity
- SystemAssignedIdentity
- UserAssignedIdentity
- DelegatedSystemAssignedIdentity
- DelegatedUserAssignedIdentity
Troubleshooting
Make sure that your default identity is that of a valid Azure resource and has all of the expected permission to access the Azure Key Vault. Once the permissions have been assigned, wait upwards to one hour for the account to stop being in revoke state. If the issue isn't resolved after more than two hours, contact customer service.
Improper Syntax Detected on the Key Vault URI Property
Reason for error
You see this error when internal validation detects that the Key Vault URI property on the Azure Cosmos DB account is different than expected.
Troubleshooting
You need to update your account's KeyVaultkeyUri property to a valid Key Vault key URI. An example of a valid Azure Key Vault's key Uri would be: "https://ContosoKeyVault.vault.azure.net/keys". It's important to mention that including the version of the key isn't needed.
Once the KeyVaultKeyUri property has been updated, wait upwards to one hour for the account to stop being in revoke state. If the issue isn't resolved after more than two hours, contact customer service.
Internal unwrapping procedure error
Reason for error
You see the error message when the Azure Cosmos DB service is unable to unwrap the key properly.
Troubleshooting
In case that either the Key Vault or the Costumer Managed Key has been recently deleted; restore the resource and wait for one hour. If the issue isn't resolved after more than 2 hours, contact customer service.
Unable to Resolve the Key Vault's DNS
Reason for error
You see the error message when the Key Vault DNS name can't be resolved. The error may indicate that there's a major issue within the Azure Key Vault service that blocks Cosmos DB from accessing your key.
Troubleshooting
If the Key Vault has been recently deleted, you need to restore it. If not, wait upwards of two hours for the account to become available again. If none of these solutions unblock the account, contact customer service.