Uploading Blobs to Blob Storage from Databricks

Question

Uploading Blobs to Blob Storage from Databricks

William Fish 0

I cannot submit files to my blob storage after making it not public. I have added my databricks IP and am using a SAS token to upload the blobs.

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Hari Babu Vattepally 3,270 Reputation points Microsoft External Staff Moderator

2025-06-11T18:18:45.4033333+00:00
Hi @William Fish,

I understand that you are not able to upload files into a private blob storage using SAS token.

In this case, you need to make sure that the SAS token has the correct permissions, for uploading files it requires of minimum of Create(c), Write(w) and Add(a)(if appending).

For example, as below command:

az storage container generate-sas \ --account-name <storage-account> \ --name <container-name> \ --permissions acw \ --expiry 2025-06-06T00:00:00Z \ --https-only \ --output tsv

If you are still facing the issue even after trying the above suggestion, check if the allowed IP address on the SAS token match the Public IP of your Databricks cluster.

However, since your storage account is Private now. You should add the outbound IP addresses of your Databricks workspace (or the public IP address ranges from your Databricks cluster) to the storage account firewall exceptions.

Go to your Storage Account → Under Security + Networking``→``Networking → Firewalls and virtual networks.

Add the Databricks IP or a Virtual Network (VNet) as appropriate

Additionally, please make sure that your SAS token is correctly formatted which you are using it in your requests properly.

Please check and confirm that the storage account endpoint (e.g., *.blob.core.windows.net) and the SAS token usage are matching. In Databricks, you should reference the storage account with the standard endpoint (unless you are using a custom DNS name).

Also, please make sure that the SAS token is valid which is not expired.

Make sure that your cluster includes the necessary libraries for Azure Blob Storage interaction. While Databricks' default Spark configurations support Blob Storage via wasbs, you may need to add the Azure Storage dependency library if any issues arise.

I hope this helps in solving the issue. If the issue still persists, please provide the screenshot with the error message. So that we can assist you better.

Please refer the below documents for the more information:

SAS tokens for your storage containers

Use Azure Blob storage Shared Access Signatures to restrict access to data in HDInsight

Please do not forget to "up-vote" whether the information provided helps you, this can be beneficial to other community members.
Hari Babu Vattepally 3,270 Reputation points Microsoft External Staff Moderator

2025-06-12T08:36:50.6333333+00:00

Hi @William Fish,

Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Venkatesan S 2,820 Reputation points Microsoft External Staff Moderator

2025-06-13T05:26:58.8566667+00:00

Hi William Fish

I wanted to follow up and see if the given answer was helpful. If this resolves your issue, please click “up-vote" for it, which could help other community members who read this thread. And, if you have any more questions, please let us know

1 answer

Your answer

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Hari Babu Vattepally 3,270 Reputation points Microsoft External Staff Moderator

2025-06-12T08:36:50.6333333+00:00

Hi @William Fish,

Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Venkatesan S 2,820 Reputation points Microsoft External Staff Moderator

2025-06-13T05:26:58.8566667+00:00

Hi William Fish

I wanted to follow up and see if the given answer was helpful. If this resolves your issue, please click “up-vote" for it, which could help other community members who read this thread. And, if you have any more questions, please let us know

Answer 1

Hi William Fish

Adding more points to above response.

According to this MS-Document,

Can you please let us know if the Databricks and the Storage account are in the same region or not?

If they are, note that whitelisting the Public IP of the databricks in Storage Account firewall would not help.
You might want to consider using Service EndPoint
This is because IP network rules have no effect on requests that originate from the same Azure region as the storage account.
- Services deployed in the same region as the storage account use private Azure IP addresses for communication. So, you can't restrict access to specific Azure services based on their public outbound IP address range.

Please let us know if the set up works with a service endpoint for Azure Storage.

You must add the Service EndPoint in the Subnet of this databricks
And also, you must allow this VNET/Subnet from the Storage Account side as well.

Hope this answer helps! please let us know if you have any further queries. I’m happy to assist you further.

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members. User's image

Share via

Uploading Blobs to Blob Storage from Databricks

1 answer

Your answer