Greetings & Welcome to Microsoft Q&A forum!
This is a known issue with Databricks storage account associated with the Managed resource group.
What is Managed Resource Group?
This resource group holds all the resources that are required by the managed application. For example, this resource group contains the virtual machines, storage accounts, and virtual networks for the solution. The customer has limited access to this resource group because the customer doesn't manage the individual resources for the managed application.
Is it possible to modify or delete the Managed Resource Group?
When we create an Azure Databricks workspace, by design it will automatically create Databricks Managed resource Group. The managed resource group must exist as this is where your cluster(s) will be created. To ensure that nothing breaks them, they are placed in a separate resource group (managed resource group) that has a super lock on it so you cannot modify anything in it.
This deny assignment is preventing you from accessing the Azure Storage Account.
Deny assignments are a type of Azure Policy assignment that explicitly denies access to a specific action in Azure. In your case, the deny assignment is blocking the action 'Microsoft.Storage/storageAccounts/listKeys/action' on the specified scope. Unfortunately, you can't override this deny assignment on the managed resource group created by Azure Databricks, even if you're the owner. This is because it's a resource managed by Databricks, and it prevents direct access to the data because it stores some system information inside the storage account. If you need to access data in the Databricks managed storage account, it's recommended to use DBFS for temporary data or mount a new storage account.
For more details refer to below links:
Azure Policy - Deny assignments: https://docs.microsoft.com/en-us/azure/role-based-access-control/deny-assignments
SO thread addressing similar issue: https://stackoverflow.com/questions/73064767/how-to-override-deny-assignment-so-that-i-can-access-the-databricks-managed-stor
Hope this helps. Do let us know if you any further queries.
If this answers your query, do click Accept Answer
and Yes
for was this answer helpful. And, if you have any further query do let us know.