Sync data between AWS S3 and azure Blob

Question

Hi Team,

I am looking for solution where data in AWS S3 and Azure Blob should be in sync so that if new files added or updated in source then it should reflected or sync in destination and vice versa both.

Please let us know whether AZ-copy will sync in this both way scenario ?
is there any tool or service available to sync data in both source and destination ?

Please help...

Answer 1

Azcopy should does this job if you have the right code but it will not work automatically or in scheduled time. And as a straightforward answer to your question, there are GUI tools that have the ability to copy /sync to/from Azure blob/AWS S3 , or to keep both in sync, for example, Gs Richcopy360 and Goodsync, also both can copy the updates files only, both can transfer between different clouds storage like from Azure blob to S3, from S3 to Azureblob, from Azure blob to Wasabi.........etc.
Hope my answer helps

Answer 2

Hello @Jayesh Sawant ,

Thanks for the question and using MS Q&A platform.

Unfortuantely, there is no specific tool or the service available to sync data in both AWS S3 and Azure Blob in Azure.

As per the research I had found third-party article which explains about the Multi-cloud connection: Transfer files between Amazon S3, Azure Blob and Google Cloud Storage using SFTP Gateway

Disclaimer: This response contains a reference to a third-party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is jhow you can be part of Q&A Volunteer Moderators

Answer 3

Hi, @Jayesh Sawant

As @PRADEEPCHEEKATLA said, there is no bi-directional synchronization tool.

I will guide you through the results of my past testing and possible approaches that can be implemented.
The following article is a verified article when I tried to do the same thing before. (Sorry it's in Japanese. Please translate)

https://dev.classmethod.jp/articles/azure-storage-azcopy-amazon-s3-datasource/

First of all, AzCopy allows him to specify S3 as the source, but not as the target.
This was also discussed in an Issue, but it seems there are no plans to implement it.

https://github.com/Azure/azure-storage-azcopy/issues/348

However, on AWS he shows how to use azure-blob-to-s3 for Blob as source and S3 as target.

https://aws.amazon.com/jp/blogs/storage/one-way-to-migrate-data-from-azure-blob-storage-to-amazon-s3/

Therefore, one way for you to achieve this without relying on third-party tools is to implement separate one-way synchronizations from AWS to Azure and from Azure to AWS. (You may also need to think about how to consider conflicts, etc.)