Share via

Azure Data Flow how to send file to SFTP

Kacper Bury 0 Reputation points
2023-03-07T15:45:07.6033333+00:00

Hello
I tried to export result of my data flow to my SFTP as single file.
I configured SFTP sink like on the screenshot
Partition option is set to "single partition".

I started pipeline with my data flow and I encoured two problems

  1. The pipeline does not stop
  2. Split files in .csv format are sent to my SFTP server instead of one single .txt file as I configured in SFTP sink
    Partition option is set to "Single partition".
    What did I do wrong ?
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,196 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. MartinJaffer-MSFT 26,061 Reputation points
    2023-03-17T01:37:19.1066667+00:00

    @Kacper Bury Hello and welcome to Microsoft Q&A

    I understand you are getting unexpected results when using DataFlow to write to SFTP. Namely getting multiple files instead of a single.

    From what you have written, I see nothing wrong. I can think of two things.

    Did you do this in "debug run" or "trigger now"/triggered run? I'm thinking maybe it ran an old version (published) of the pipeline instead of the one with intended settings (debug).

    The other thing to try, is setting file name options: output to single file

    User's image

    Ideally, it should have only writen one data file with single partition. Was the other file named like _success ?

    0 comments