SSIS write a 60Gb data into Amazon S3

CzarR 296 Reputation points
2022-05-04T01:13:34.023+00:00

Hi, I am extracting data from a table that has 28 million rows and about 60Gb in size. We have CozyRoc suite. One way is to download the file to a local folder and then use CozyROC SFTP from there to move it to S3. Rather than downloading it to a local folder and moving it to S3, is there a way to directly write into S3?

We use SSH connection manager to connect to S3.

Please suggest. Thanks in advance.

SQL Server Integration Services
SQL Server Integration Services
A Microsoft platform for building enterprise-level data integration and data transformations solutions.
2,508 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. ZoeHui-MSFT 34,996 Reputation points
    2022-05-04T02:31:41.04+00:00

    Hi @CzarR ,

    Not familiar with Amazon S3 and the 3rd party tool.

    Maybe you could try with script task to load data to S3.

    Check this to see if it is helpful.

    Regards,

    Zoe


    If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    1 person found this answer helpful.
    0 comments No comments