How we can transfer files from blob storage to s3 bucket using ADF

Question

file need to be transferred which is stored in azure blob storage to amazon s3 bucket , how we can achieve this using ADF as currently using ADF copy activity s3 is not allowed as sink.

Answer 1

Thanks for posting your question in the Microsoft Q&A forum.

On a high level, below are the steps:

• Create a Python script to transfer file from BLOB to S3 (Script is given at the end of this post).
• Create an Azure Batch account and configure the batch pool.
• Create an ADF pipeline with a custom activity, and connect to Azure batch to run the data transfer script.

https://medium.com/litmus7/file-transfer-from-azure-blob-to-aws-s3-step-by-step-guide-9be4b033b8ea

** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful **

Answer 2

@A. Kumar Welcome to Microsoft Q&A Forum, Thank you for posting your query here!
Just wanted you to point you to this thread , I think its the exact ask .

Additional information: How to copy data from Azure Blob Storage to Amazon S3 using code
Migrate data from Microsoft Azure Blob to Amazon S3 by using Rclone

Please let us know if you have any further queries. I’m happy to assist you further.    

---

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 3

Try Rclone or Azcopy to migrate from Azure blob to AWS S3 , Also you can try tools like CloudFuze or Gs Richcopy 360 to migrate your files directly from Azure blob to AWS S3 . search all tools and try the trial version first