Databricks Support Azure Keyvault backed Secret Scope where Azure Keyvault uses RBAC instead of Access Policy.

Question

Recently a new feature was released in Azure Key Vault where access to the vault can be managed via Azure RBAC roles (instead of access policies). An AKV instance can configured to use either RBAC or Access Policy (but not both at the same time). https://learn.microsoft.com/en-us/azure/key-vault/general/rbac-guide?tabs=azure-cli

Azure Databricks has a feature where a Secret Scope can be backed by AKV https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes#--create-an-azure-key-vault-backed-secret-scope.

I am having trouble getting this Databricks Secret Scope feature to work with AKV configured to use RBAC for Authorisation. While the creation of the Secret Scope works when I try to list the secrets it throws an error (as per the screenshot below). I created a second Secret Scope that uses an AKV configured to use Access Policies and that works without any issue. Screenshot of testing below.

Are AKV that use RBAC not supported for Databricks Secret Scopes?

Answer 1

Hello @P ,

Thanks for the question and using MS Q&A platform.

By default, Azure Databricks have an associated Enterprise application called "AzureDatabricks".

For more details, refer to Enable customer-managed keys for managed services.

How to get the Azure Databricks application from Azure portal?

• In Azure Active Directory, select Enterprise Applications from the sidebar menu.
• Search for AzureDatabricks and click the Enterprise application in the results.

As per the repro, when I used Azure role-based access control in Azure Key Vault noticed the same error as shown above:

To resolve this issue, you need to Add role assignment to the AzureDatabricks application with role Key Vault Administrator.

After adding the Key Vault Administrator role to the AzureDatabricks application able to list the scopes inside the key vault.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

For MS stakeholders following this:

• Is proper key vault RBAC support for Databricks planned at all?
  • Currently the user/identity creating a key vault-backed scope must have access to key vaults' access policy list and existing RBAC-based access control is being ignored
  • Scope creation always adds AzureDatabricks enterprise app into access policy list even if key vault is configured to use RBAC

The current accepted answer is just a workaround and the underlying issues still exists on Databricks

TLDR: Azure Databricks does not work nicely with RBAC-based key vault access. Will there be changes for that?

Answer 3

PSA: RBAC-based key vault access control is not support by MSFT ref: https://github.com/MicrosoftDocs/azure-docs/issues/113076 MSFT has actively refused to answer clarifications about the changes they did with the documentation. E.g. "How are support tickets handled if customer is using RBAC-based key vaults for databricks secret scopes?" is still being unanswered. If you need key vault's RBAC-based access control I suggest contacting presales support before implementing anything.