amazon s3 access points data factory transfer

Question

I'm trying to transfer data over from AmazonS3 to my Azure Data Lake. I'm currently given access to the s3 bucket via an access point.

https://aws.amazon.com/s3/features/access-points/

When I write python it looks like this:

AWSAccessKeyId='AKIAJQSDEGOZWTVPUETQ'
AWSSecretKey='xxxxx'

Bucket="arn:aws:s3:us-west-2:xxxxx:accesspoint/datafeed", 
Prefix="dplp-aea10a31/"

client = boto3.client('s3', aws_access_key_id=AWSAccessKeyId, aws_secret_access_key=AWSSecretKey, region_name='us-east-1' )

dict_objects = client.list_objects_v2(
  Bucket="arn:aws:s3:us-west-2:xxxxx:accesspoint/datafeed", 
  Prefix="dplp-aea10a31/"
)

The problem I have now is I have to download the file and read it. I'm looking to transfer it to my data lake to make it easier to deal with.

I don't see any way to do this thru Azure Synapse / Data Lake at first look.

Answer 1

Hello @B K ,

Thanks for the question and using MS Q&A platform.

There are couple of ways to access Amazon Simple Storage Service in Azure.

• Copy data from Amazon Simple Storage Service using Azure Data Factory or Synapse Analytics to Azure Data Lake Storage Gen2 and then you can access ADLS gen2 using SQL or Spark pools. Refer to : Explore the Manage Hub in Synapse Studio to provision and secure resources
• Unloading from AWS S3 & loading into Azure Synapse Table via Databricks Hope this will help. Please let us know if any further queries.

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators