Copy Data Activity - Additional Columns - More Columns Than Expected

b-sherbert 1 Reputation point
2022-11-18T17:57:37.47+00:00

I am new to ADF and the soloution is likely simple but I have not been able to find it anywhere.

I am attempting to copy csv files from a SFTP server to an Azure container. I am using the "additional columns" feature in the copy activity to add the name of the source files as well as the load date (using the system variable @pipeline().TriggerTime). The first row is used as a header in both source and sink datasets. I am getting an error for "more columns than expected" upon execution. The pipeline runs fine without the additional columns. What am I missing?

262003-screenshot-20221118-104631.png262004-screenshot-20221118-104710.png261993-screenshot-20221118-104921.png261970-screenshot-20221118-105410.png

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,532 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Nasreen Akter 10,736 Reputation points
    2022-11-18T18:57:11.777+00:00

    Hi @b-sherbert ,

    Thank you for the ask. I was wondering about the schema you have for the SINK dataset. Did you set a schema for the SFTP_to_DW dataset. If you have, please add the two columns in your SINK-Schema. Hope that helps, thanks!