sink parquet

Question

hello,
I am using a copy activity.
source --> dataset - a dummy .csv file with one column and one value in column.
The source has additional columns which I added manually. This is the setting at the bottom of the source tab
sink --> dataset which is in .parquet

everytime I run the copy activity, the sink .parquet file gets replaced with the new .parquet and so the previous data disappears.
Question.
How is it possible to get multiple lines in my sink folder. perhaps how is it possible to have a separate parquet file for each run in sink?
Thanks

Answer 1

Hi @arkiboys ,

Appending to the same file is not possible with copy activity.

Alternatively, as you asked, you can create a new file everytime. For that create a parameter for filename in sink dataset and pass an expression like below from the copy activity sink.

@markus.bohland@hotmail.de ('filename_',formatDateTime(utcNow(),'ddMMyyyyhhmmss'),'.csv')

This will create a new file everytime your pipeline is run.

Hope this helps. Let us know for any question.