AWS DataSync S3 Bucket Copy Files To Azure Storage Account Container

Question

Hi Expert,

I had an issue AWS DataSync S3 Bucket Copy Files To Azure Storage Account Container.

I setup AWS DataSync for S3 Bucket (Location / Source)

Copy file to Azure Storage Account Container (destination)—it already have the permission to access shared SAS Token (with Read , Write , List , Delete)

when i run the task, it hit error Task failed due to a transient error. Please re-run the task and contact AWS support if the problem persists.

that is the log; i can get it on aws datasync.

how i can fix it ?

thanks.

Answer 1

@Kelvin Shee Welcome to Microsoft Q&A Forum, Thank you for posting your query!

For better understanding issue: Can you please share the screenshot of the error message?

• Based on the error message : This means that the error was temporary and you can try running the task again.
• Check the SAS Token Permissions: SAS token permissions for container-level access (Ensure that the SAS token you are using has the necessary permissions (Read, Write, List, and Delete)
• Also cross verify the Network configuration and check if Network firewall is blocking the connectionetween AWS DataSync and the Azure Storage Account.
• Review the detailed logs provided by AWS DataSync for any specific error messages or codes that can give more insight into the issue. If the problem persists, consider reaching out to AWS support for further assistance, I understand the issue you're facing, but since AWS tools are outside my expertise,

I would recommed try using Azcopy tool for data transfer from AWS to Azure Storage (Copy data from Amazon S3 to Azure Storage by using AzCopy)
Troubleshooting issues with AWS DataSync transfers

Addditional information:
AWS DataSync: If you're moving data out of S3 or EFS, you can use AWS DataSync to transfer data to Azure Blob Storage or Azure Files.
Use Azure Migrate with private endpoints - Azure Migrate

Answer 2

@Kelvin Shee I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this!

Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer. Accepted answers show up at the top, resulting in improved discoverability for others.

Issue: AWS DataSync fails to copy files from an S3 bucket to an Azure Storage Account container, despite having the correct SAS token permissions (Read, Write, List, Delete), and receiving the error: "Task failed due to a transient error. Please re-run the task and contact AWS support if the problem persists"

Solution: The DataSync agent requires 8GB of memory to perform the task of copying files from an AWS S3 bucket to an Azure Storage Account container.

If your issue remains unresolved or have further questions, please let us know in the comments how we can assist. We are here to help you and strive to make your experience better and greatly value your feedback.