Azure Data Factory - Foreach for multiple CSV

Shah, Sapana N 1

Hello, I need to load multiple csv files from ADLS to SQL Db. For pipeline I am using lookupfiles to get the file list and than passing it to foreach. Within foreach I am using copydata. But i am getting error as each csv file has two extra lines of data before header starts. If i use skip line count within copydata than pipeline is throwing an error. Copy data for the same source and sink works without an error for single file but not within foreach. Any suggestion please ?

KranthiPakala-MSFT 46,442 Reputation points Microsoft Employee

2021-03-04T07:03:54.767+00:00

Hi @Shah, Sapana N ,

Welcome to Microsoft Q&A forum and thanks for reaching out. Could you please share the error message you are receiving?

Thanks
KranthiPakala-MSFT 46,442 Reputation points Microsoft Employee

2021-03-05T22:46:52.31+00:00

Hi @Shah, Sapana N ,

Following up to see if you have got a chance to see my previous comment? If so could you please share the error message you are receiving as well the failed pipeline runID and failed activity runID?

Thanks
sapana shah 1 Reputation point

2021-03-08T14:37:07.81+00:00

Thanks for your response ! Sorry for late reply but there were deadlines at work. Issue is resolved now. But just wondering for future Q&A, will you be able to find the error with runID itself ? Asking as I am new to ADF. Thanks again.
KranthiPakala-MSFT 46,442 Reputation points Microsoft Employee

2021-03-09T17:50:12.927+00:00

Hi @sapana shah ,

Thanks for your response and glad to know that the issue was resolved. Would you please share the resolution steps here with community as it could be beneficial for others who read this thread.

Also for future Q&A, I would suggest to share the complete error message along with your failed pipeline runID and activity runID so that It would help to narrow down the root cause analysis and also reduce the resolution time.

Share via

Azure Data Factory - Foreach for multiple CSV