How we can transfer files from blob storage to s3 bucket using ADF

A. Kumar 0 Reputation points
2024-05-23T14:04:46.7333333+00:00

file need to be transferred which is stored in azure blob storage to amazon s3 bucket , how we can achieve this using ADF as currently using ADF copy activity s3 is not allowed as sink.

Azure Files
Azure Files
An Azure service that offers file shares in the cloud.
1,203 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. Vahid Ghafarpour 19,865 Reputation points
    2024-05-23T18:12:57.2366667+00:00

    Thanks for posting your question in the Microsoft Q&A forum.

    On a high level, below are the steps:

    • Create a Python script to transfer file from BLOB to S3 (Script is given at the end of this post).
    • Create an Azure Batch account and configure the batch pool.
    • Create an ADF pipeline with a custom activity, and connect to Azure batch to run the data transfer script.

    https://medium.com/litmus7/file-transfer-from-azure-blob-to-aws-s3-step-by-step-guide-9be4b033b8ea

    ** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful **

    0 comments No comments

  2. Sumarigo-MSFT 44,506 Reputation points Microsoft Employee
    2024-05-24T10:14:53.4466667+00:00

    @A. Kumar Welcome to Microsoft Q&A Forum, Thank you for posting your query here!
    Just wanted you to point you to this thread , I think its the exact ask .

    Additional information: How to copy data from Azure Blob Storage to Amazon S3 using code
    Migrate data from Microsoft Azure Blob to Amazon S3 by using Rclone

    Please let us know if you have any further queries. I’m happy to assist you further.    


    Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    0 comments No comments