Databricks - Cannot activate File Events for External Location / ADLS V2

Question

Databricks - Cannot activate File Events for External Location / ADLS V2

Dimitry Nechaev 50

I've followed the book on Databricks for creating external location for Azure Data Lake Storage (ADLS V2) using account connector.

I've granted all required permissions to the connector:

I've created a "stock" container on that above mentioned "devtyremeshare" storage account:

Configured storage account as recommended

Databricks can list and read files and folders:

But fails to activate File Events. It reports missing permission to write into the storage account, even after I created an additional custom role with 'Microsoft.Storage/storageAccounts/write' permission on the subscription level and assigned to the app connector (which is 4d2fd0d9-2339-498a-b058-990c13b55cf9 below).

It feels to me that is not the IAM but something else.

Job, if hooking on file events, will error exactly the same.

Event Grid Resource Provider is activated on the subscription level.

Storage account is public w/o firewall.

Databricks is default setup with public IP.

Please help.

Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-07-17T02:24:19.02+00:00
Hi Dimitry Nechaev
Thanks for all the detailed info and screenshots - that really helps in narrowing down the issue.

From what you have described, Databricks has the right access to read/list files from your ADLS Gen2 container, but it fails when trying to enable File Events. The error message indicates a permission issue while creating the Event Grid subscription, even though you’ve granted the Microsoft.Storage/storageAccounts/write permission at the subscription level.

Here’s what might be causing it and how to fix it:

Things to Review

Role Assignment Scope Matters

Although you've assigned the required permissions at the subscription level, Event Grid operations like eventSubscriptions/write often require permissions to be set specifically at the storage account level. Please make sure the Databricks account connector (SPN or managed identity) has the following roles assigned directly on the storage account: Storage Blob Data Contributor

Storage Queue Data Contributor

Storage Account Contributor OR a custom role that includes:

Microsoft.Storage/storageAccounts/write

Microsoft.EventGrid/eventSubscriptions/* Microsoft.Storage/storageAccounts/blobServices/containers/write

Event Grid Provider Setup

You’ve already confirmed that the Microsoft.EventGrid resource provider is registered — that’s good. Just in case: make sure the Databricks identity being used is the one that needs permissions to create Event Grid subscriptions.

Network Access

Since your storage account has no firewall restrictions, access shouldn’t be an issue. Still, it’s worth double-checking that the account allows either all networks or includes Databricks IP ranges if you’re on a public setup.

What you can try next

Reassign the required permissions at the storage account level, not just at the subscription.

Use this CLI command to confirm what roles are applied:
az role assignment list --assignee <Databricks-SPN-Object-ID> --scope /subscriptions/<subId>/resourceGroups/<RG>/providers/Microsoft.Storage/storageAccounts/<StorageName>
If all permissions look correct and it still fails, feel free to share the Request ID or full error message for the failed event. That can help isolate if it’s purely RBAC-related or something else.

Let me know how it goes or if you'd like help reviewing the roles from your side. Thanks again for taking the time to document your setup so clearly - it really helps move things forward faster.
I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-07-17T10:15:23.28+00:00

Dimitry Nechaev We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Dimitry Nechaev 50

Hi Smaran

The roles appears to be the same as on the provided screenshot, except only on the storage account level

User's image

So all roles that you mentioned were already assigned by the book.

[
  {
    "condition": null,
    "conditionVersion": null,
    "createdBy": "ad24b16e-c539-4ff4-8f5e-5fef5d1e4b83",
    "createdOn": "2025-07-11T05:02:48.045505+00:00",
    "delegatedManagedIdentityResourceId": null,
    "description": null,
    "id": "/subscriptions/<sub-id>/resourceGroups/dev-shared-rg/providers/Microsoft.Storage/storageAccounts/devtyremeshare/providers/Microsoft.Authorization/roleAssignments/7ff98d32-25bb-4de5-a031-15e763b333f4",
    "name": "7ff98d32-25bb-4de5-a031-15e763b333f4",
    "principalId": "4d2fd0d9-2339-498a-b058-990c13b55cf9",
    "principalName": "73851434-9272-4fbe-985f-fb42943c22bb",
    "principalType": "ServicePrincipal",
    "resourceGroup": "dev-shared-rg",
    "roleDefinitionId": "/subscriptions/<sub-id>/providers/Microsoft.Authorization/roleDefinitions/17d1049b-9a84-46fb-8f53-869881c3d3ab",
    "roleDefinitionName": "Storage Account Contributor",
    "scope": "/subscriptions/<sub-id>/resourceGroups/dev-shared-rg/providers/Microsoft.Storage/storageAccounts/devtyremeshare",
    "type": "Microsoft.Authorization/roleAssignments",
    "updatedBy": "ad24b16e-c539-4ff4-8f5e-5fef5d1e4b83",
    "updatedOn": "2025-07-11T05:02:48.045505+00:00"
  },
  {
    "condition": null,
    "conditionVersion": null,
    "createdBy": "ad24b16e-c539-4ff4-8f5e-5fef5d1e4b83",
    "createdOn": "2025-07-11T05:03:14.855498+00:00",
    "delegatedManagedIdentityResourceId": null,
    "description": null,
    "id": "/subscriptions/<sub-id>/resourceGroups/dev-shared-rg/providers/Microsoft.Storage/storageAccounts/devtyremeshare/providers/Microsoft.Authorization/roleAssignments/3ea5918e-aae2-4ed2-8c44-111c8cc277b7",
    "name": "3ea5918e-aae2-4ed2-8c44-111c8cc277b7",
    "principalId": "4d2fd0d9-2339-498a-b058-990c13b55cf9",
    "principalName": "73851434-9272-4fbe-985f-fb42943c22bb",
    "principalType": "ServicePrincipal",
    "resourceGroup": "dev-shared-rg",
    "roleDefinitionId": "/subscriptions/<sub-id>/providers/Microsoft.Authorization/roleDefinitions/ba92f5b4-2d11-453d-a403-e96b0029c9fe",
    "roleDefinitionName": "Storage Blob Data Contributor",
    "scope": "/subscriptions/<sub-id>/resourceGroups/dev-shared-rg/providers/Microsoft.Storage/storageAccounts/devtyremeshare",
    "type": "Microsoft.Authorization/roleAssignments",
    "updatedBy": "ad24b16e-c539-4ff4-8f5e-5fef5d1e4b83",
    "updatedOn": "2025-07-11T05:03:14.855498+00:00"
  },
  {
    "condition": null,
    "conditionVersion": null,
    "createdBy": "ad24b16e-c539-4ff4-8f5e-5fef5d1e4b83",
    "createdOn": "2025-07-11T05:03:43.816125+00:00",
    "delegatedManagedIdentityResourceId": null,
    "description": null,
    "id": "/subscriptions/<sub-id>/resourceGroups/dev-shared-rg/providers/Microsoft.Storage/storageAccounts/devtyremeshare/providers/Microsoft.Authorization/roleAssignments/0acdc6c0-306d-41ab-8a7e-ad645d761db4",
    "name": "0acdc6c0-306d-41ab-8a7e-ad645d761db4",
    "principalId": "4d2fd0d9-2339-498a-b058-990c13b55cf9",
    "principalName": "73851434-9272-4fbe-985f-fb42943c22bb",
    "principalType": "ServicePrincipal",
    "resourceGroup": "dev-shared-rg",
    "roleDefinitionId": "/subscriptions/<sub-id>/providers/Microsoft.Authorization/roleDefinitions/974c5e8b-45b9-4653-ba55-5f855dd0fb88",
    "roleDefinitionName": "Storage Queue Data Contributor",
    "scope": "/subscriptions/<sub-id>/resourceGroups/dev-shared-rg/providers/Microsoft.Storage/storageAccounts/devtyremeshare",
    "type": "Microsoft.Authorization/roleAssignments",
    "updatedBy": "ad24b16e-c539-4ff4-8f5e-5fef5d1e4b83",
    "updatedOn": "2025-07-11T05:03:43.816125+00:00"
  }
]

Dimitry Nechaev 50 Reputation points

2025-07-18T01:02:01.28+00:00

Also checked Databricks.

Target external location uses correct connector:

And the connector is bound to the correct Azure indentity:
Dimitry Nechaev 50 Reputation points

2025-07-21T01:06:17.6866667+00:00

@Smaran Thoomu can you please look into the above
Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-07-21T02:31:45.06+00:00
Hi Dimitry Nechaev
Thanks for the detailed update and for verifying the role assignments at the storage account level - that helps rule out several common misconfigurations.

Based on the full context and screenshots, including:

Correct connector-to-identity mapping in Databricks

All necessary roles (Storage Account Contributor, Storage Blob Data Contributor, Storage Queue Data Contributor) being present

Storage account publicly accessible (no firewall restrictions)

Event Grid Resource Provider being enabled

We’re now likely dealing with either a missing permission specific to Event Grid subscription creation, or a limitation in how the connector identity is being interpreted by Databricks during file event activation.

Next Steps to Try:

Add Explicit Event Grid Permissions

Even with Storage Account Contributor, sometimes Databricks requires explicit permission for Event Grid operations. Please add a custom role or built-in "EventGrid EventSubscription Contributor" role at the storage account level for the connector SPN:

Role Name: EventGrid EventSubscription Contributor Permission: Microsoft.EventGrid/eventSubscriptions/*

You can assign it via Azure CLI:

az role assignment create \ --assignee <ServicePrincipalObjectID> \ --role "EventGrid EventSubscription Contributor" \ --scope /subscriptions/<sub-id>/resourceGroups/<rg-name>/providers/Microsoft.Storage/storageAccounts/<storage-name>

Verify Principal Used for Event Subscription

Even if the external location uses the connector correctly, internally Databricks might attempt to use the workspace managed identity or Unity Catalog metastore identity (if used). Ensure the correct identity has the needed Event Grid permissions.

If you're unsure, please share the exact error message or Request ID when file event activation fails — that will help us trace which identity attempted the operation.

Confirm Databricks Runtime Version

There were known issues with File Events activation in certain earlier runtime versions. Please confirm that your workspace is running an up-to-date Databricks Runtime 13.x or higher.

Let me know once the EventGrid role is added and if the issue persists - we can further investigate with a specific request correlation ID from the failure.

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.
Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-07-22T11:37:38.69+00:00

Dimitry Nechaev We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Dimitry Nechaev 50 Reputation points

2025-07-23T04:39:37.47+00:00

@Smaran Thoomu I have responded 3 times, what do you mean you didn't hear from me?
Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-07-23T07:04:07.84+00:00

Dimitry Nechaev Thank you for your continued engagement and for highlighting that - I sincerely appreciate your attention to detail and the updates you've shared.

You're absolutely right, and I acknowledge that your responses were received. I should have been more clear in referencing the current status of the troubleshooting steps we previously discussed. To help us move forward and isolate the root cause, could you please share the details mentioned in the private message? This will allow us to address the issue more effectively.

Dimitry Nechaev 50

Hi @Smaran Thoomu

I've followed your guide and unfortunately I had no luck.

This is a current set of premissions on the storage account

User's image

And this is the "Test connection" error

File Events Read Failed
Provisioning has failed. Failed to provision file events resources during eventSubscription.create operation. Please ensure the permission settings are correctly set. Original error from the cloud provider: Status code 403, "{"error":{"code":"LinkedAuthorizationFailed","message":"The client '4d2fd0d9-2339-498a-b058-990c13b55cf9' with object id '4d2fd0d9-2339-498a-b058-990c13b55cf9' has permission to perform action 'Microsoft.EventGrid/eventSubscriptions/write' on scope '/subscriptions/3e8044a3-2e4f-4c5e-a0c1-770a2b691ee4/resourceGroups/shared-resources/providers/Microsoft.Storage/storageAccounts/devtyremeshare/providers/Microsoft.EventGrid/eventSubscriptions/csms-subscription-1abbd78f-ef2c-497c-b392-260c0aa89514'; however, it does not have permission to perform action(s) 'Microsoft.Storage/storageAccounts/write' on the linked scope(s) '/subscriptions/3e8044a3-2e4f-4c5e-a0c1-770a2b691ee4/resourceGroups/shared-resources/providers/Microsoft.Storage/storageAccounts/devtyremeshare/' (respectively) or the linked scope(s) are invalid."}}"

To unwrap on this confusing part:


The client '4d2fd0d9-2339-498a-b058-990c13b55cf9' with object id '4d2fd0d9-2339-498a-b058-990c13b55cf9' ... does not have permission to 'Microsoft.Storage/storageAccounts/write' on the linked scope(s) '/subscriptions/3e8044a3-2e4f-4c5e-a0c1-770a2b691ee4/resourceGroups/shared-resources/providers/Microsoft.Storage/storageAccounts/devtyremeshare/'

This guid is of the intended connector identity: User's image

And how I understand the message, it says that the connector doesn't have permission to write to the account when it has the same permissions as myself for writing, and I have no prob writing myself. It makes no sense.

As of 'Microsoft.EventGrid/eventSubscriptions/write' permission, I don't think it complaints on it, as it says "has permission to perform action". So it can write to the event grid.

Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-08-01T11:15:20.7133333+00:00

Dimitry Nechaev Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-08-04T19:54:24.54+00:00

Dimitry Nechaev Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer accepted by question author

0 additional answers

Your answer

Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-07-17T10:15:23.28+00:00

Dimitry Nechaev We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Dimitry Nechaev 50 Reputation points

2025-07-18T01:02:01.28+00:00

Also checked Databricks.

Target external location uses correct connector:

And the connector is bound to the correct Azure indentity:
Dimitry Nechaev 50 Reputation points

2025-07-21T01:06:17.6866667+00:00

@Smaran Thoomu can you please look into the above
Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-07-22T11:37:38.69+00:00

Dimitry Nechaev We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Dimitry Nechaev 50 Reputation points

2025-07-23T04:39:37.47+00:00

@Smaran Thoomu I have responded 3 times, what do you mean you didn't hear from me?
Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-07-23T07:04:07.84+00:00

Dimitry Nechaev Thank you for your continued engagement and for highlighting that - I sincerely appreciate your attention to detail and the updates you've shared.

You're absolutely right, and I acknowledge that your responses were received. I should have been more clear in referencing the current status of the troubleshooting steps we previously discussed. To help us move forward and isolate the root cause, could you please share the details mentioned in the private message? This will allow us to address the issue more effectively.
Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-08-01T11:15:20.7133333+00:00

Dimitry Nechaev Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Smaran Thoomu 32,525 Reputation points Microsoft External Staff Moderator

2025-08-04T19:54:24.54+00:00

Dimitry Nechaev Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

Hi Dimitry Nechaev

Thanks for all the detailed info and your patience throughout the troubleshooting.

Following your updates, I reached out to the internal team to take a closer look. After reviewing the setup with your configuration, the issue was successfully resolved by reconfiguring the connector setup from scratch.

It appears that while all permissions (including Microsoft.Storage/storageAccounts/write and Microsoft.EventGrid/eventSubscriptions/write) were present, there might have been a transient inconsistency in how the identity or permissions were recognized by the backend during provisioning. A full teardown and recreation of the external location and connector resolved the conflict.

Resolution Steps Taken:

Reconfigured the external location setup using the same connector identity.
Reassigned roles explicitly at the storage account scope again (Storage Account Contributor, Storage Queue Data Contributor, and Storage Blob Data Contributor).
Verified EventGrid permissions and refreshed the test connection.
File Events were activated successfully post-reconfiguration.

If others face a similar error (LinkedAuthorizationFailed despite correct role assignments), a full re-setup of the connector and location is worth trying, especially if the roles were changed after the connector was initially configured.

Let us know if you need anything else - and appreciate your persistence on this!

I hope this information helps. Please do let us know if you have any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

Databricks - Cannot activate File Events for External Location / ADLS V2

0 additional answers

Your answer