Share via

SSIS write a 60Gb data into Amazon S3

CzarR 316 Reputation points
2022-05-04T01:13:34.023+00:00

Hi, I am extracting data from a table that has 28 million rows and about 60Gb in size. We have CozyRoc suite. One way is to download the file to a local folder and then use CozyROC SFTP from there to move it to S3. Rather than downloading it to a local folder and moving it to S3, is there a way to directly write into S3?

We use SSH connection manager to connect to S3.

Please suggest. Thanks in advance.

SQL Server Integration Services
SQL Server Integration Services
A Microsoft platform for building enterprise-level data integration and data transformations solutions.
2,705 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. ZoeHui-MSFT 41,496 Reputation points
    2022-05-04T02:31:41.04+00:00

    Hi @CzarR ,

    Not familiar with Amazon S3 and the 3rd party tool.

    Maybe you could try with script task to load data to S3.

    Check this to see if it is helpful.

    Regards，

    Zoe

    If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    1 person found this answer helpful.
    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.