Data Factory Loading multiple Files. Ignore if column doesnt exist

Debbie Edwards 521 Reputation points
2020-11-05T12:54:09.22+00:00

Im loading multiple csv files using a wildcard file path and i have created parameters for the fileSystem, Directory and Filename (the csvs are in a datalake)

A few of them have a couple of columns missing. is there any way in the copy activity to continue and load and just ignore the missing columns>

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,564 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. HimanshuSinha-msft 19,471 Reputation points Microsoft Employee
    2020-11-06T23:34:27.707+00:00

    Hello @Debbie Edwards ,

    Thanks for the ask .

    The fault tolrenace option will not help , here , I think you can try the option called out here .
    https://learn.microsoft.com/en-us/answers/questions/152741/data-factory-loading-multiple-csv-files-is-there-a.html?childToView=154786#answer-154786

    Thanks
    Himanshu

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.