How to upload Files from on-prem to S3 bucket using ADF

Khon 0 Reputation points
2023-02-12T15:02:56.09+00:00

Hello everyone,

We have a requirement to upload and download files between S3 bucket and on-Prem using ADF preferably. I was able to download files using S3 connector in ADF but I don’t see any direct connector for file uploading from onprem to S3 . We have self hosted IR configured and we are using site to site VPN to connect on-prem with Azure.

Any suggestions , how to implement this use case (File upload to S3from onprem) with Azure services, preferably ADF .

Thanks

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,640 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Luke Murray 11,436 Reputation points MVP Volunteer Moderator
    2023-02-13T03:14:37.4466667+00:00

    Hi, Khon

    S3 buckets can be the source (ie FROM S3 to Azure Storage) but not the sink (destination) when using ADF.

    https://learn.microsoft.com/en-us/azure/data-factory/connector-overview

    SFTP is allowed as a source and a sink, you could look at options around enabling SFTP support on the S3 bucket and connect that way.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.