Add additional Row when copying files

Question

Hello, ADLS has a parquet file, using copy activity to move files from one location to another. When copying I want to add an additional column in the sink. Actual data from Source Plus one column like a header at the top. ![11423-images.png][1] [1]: /api/attachments/11423-images.png?platform=QnA Please recommend me on the best approach.

Answer 1

Hi anonymous user,

Sorry for the delay in response. Unfortunately there is no out of box feature to add a row while copying files. I would recommend you to please provide a feedback in ADF user voice forum : https://feedback.azure.com/forums/270578-azure-data-factory . All the feedback shared in this forum are actively monitored and reviewed by ADF engineering team. Also please do share the feedback link here once it is created, as it would help others with similar idea to up-vote and/or comment on the suggestion/feedback.

One possible alternate could be to use Custom activity or a Function activity and write your own logic to achieve this.

----------

Thank you
Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.

Answer 2

Idea Submitted here - https://feedback.azure.com/forums/270578-data-factory/suggestions/41000581-additional-row-feature-in-copy-activity

Answer 3

@ChiragMishraMSFT-1092 - Sorry for the confusion, I have to add additional row in the sink.

Answer 4

Hi @Maaran-7752,

In addition to copying data from source data store to sink, you can also configure to add additional data columns to copy along to sink. For example:

• When copy from file-based source, store the relative file path as an additional column to trace from which file the data comes from.
• Add a column with ADF expression, to attach ADF system variables like pipeline name/pipeline id, or store other dynamic value from upstream activity's output.
• Add a column with static value to meet your downstream consumption need.

You can find the following configuration on copy activity source tab:

Ref - Add additional columns during copy

Hope this helps.