Azure Machine Learning workspace cannot access Datastore, Container Registry

Question

Hi,

I have created an Azure Machine Learning workspace, giving it a user-assigned identity. This identity has both a contributor role over the whole resource group, and a Key Vault Secrets Officer role over the key vault used by the AML workspace

It was working fine and I was able to run ML pipelines, until recently, Now the AML workspace can no longer access its default datastore, neither the container registry. The error I get is the following (see screenshot):

Credential Service error due to user error.This can occur if the system assigned managed identity of the workspace does not have access to the key vault. Please add workspace managed identity as Contributor for the key vault associated with the workspace. Please make sure that you are passing valid secret names and that the keyvault https://keyvaultURL

The issue is that, the user-assigned identity of the workspace has both roles I mentioned and should be able to access the vault - it also worked in the past, and I do not know what causes this, since the managed identity was not modified.

What could have caused this issue and what fix can I implement? Deleting the workspace is not an option for now, as I have some Machine Learning real-time endpoints which I use

Thanks

Answer 1

@Vengathesa Sarma, Satya Could you try to assign another UAI with contributor access to the keyvault? Looking at the table mentioned in this link, I see the keyvault needs contributor + additional roles.

I am not sure what could have caused this if there is no change in UAI but you could check if there is any policy that is added that could restrict access to the workspace or any change in scope or resource group of the resources with a move operation.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 2

Posting OP resolution as answer.

Issue: Following issue seen in ML portal.

Resolution:

User ended up having to delete the workspace and its vault and recreate them to fix the issue.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.