Upload file directly to Azure Blob Storage from URL with python

Andy Quang Anh Pham 21 Reputation points
2022-04-17T03:35:01.327+00:00

I'm trying to upload file to Blob storage from the URL: https://datasets.imdbws.com/title.akas.tsv.gz without having to download to an intermediary storage location.
Can I accomplish that with Databricks notebook in Python?

I looked into the link: https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-python?tabs=environment-variable-linux#upload-blobs-to-a-container
but it's not clear of dealing with URL.

There was an older version of BlobServiceClient which is BlockBlobService has the solution as below example but I couldn't find the similar one for BlobServiceClient

from azure.storage.blob import BlockBlobService, PublicAccess
from azure.storage.blob.models import Blob

def run_sample():

block_blob_service = BlockBlobService(account_name='your_name', account_key='your_key')    
container_name ='t1s'    
block_blob_service.copy_blob(container_name,'remoteURL.pdf','https://media.readthedocs.org/pdf/azure-storage/v0.20.3/azure-storage.pdf')    

if name == 'main':
run_sample()

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,547 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,018 questions
0 comments No comments
{count} votes

Accepted answer
  1. Sumarigo-MSFT 44,506 Reputation points Microsoft Employee
    2022-04-19T03:23:38.14+00:00

    @Andy Quang Anh Pham Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

    You can perform this scenario by using CopyBlobFromUrl (service doc) as you did in previous versions. In Python Track2 that is done with the BlobClient.start_copy_from_url() method.

    See our sample for this here. This sample even uses an external URL as the customer is looking to do.
    azure-sdk-for-python/blob_samples_copy_blob.py at main · Azure/azure-sdk-for-python (github.com)

    Please let us know if you have any further queries. I’m happy to assist you further.

    ----------

    Please do not forget to194017-screenshot-2021-12-10-121802.png and “up-vote!” wherever the information provided helps you, this can be beneficial to other community members.


0 additional answers

Sort by: Most helpful