Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,197 questions
ADF Data Flow failing with Error code : DFExecutorUserError
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 6%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EUC%3C/text%3E%3C/svg%3E)
Utsav Chanda
71
Reputation points
I am executing a pipeline with a Data Flow Activity that reads and writes parquet files organized in a partitioned folder structure in ADLS Gen 1. The Data Flow and Pipeline validation are fine. I can also see data in Sink Data Preview. However, on running the pipeline, I am repeatedly getting the below error
Can anybody suggest what is the fix for this
{count} votes
-
ShaikMaheer-MSFT 38,401 Reputation points Microsoft Employee2021-09-08T12:39:34.623+00:00 Hi @Utsav Chanda ,
Thank you for posting query in Microsoft Q&A Platform.
Error message doesn't seems to be much helpful. Below might be possible reasons for this.
- Data preview will only take few rows as sample data and perform transformations on it, But when you actually run data flow it will try to perform transformations on whole data. So there might some issue with data itself.
- Could you please try increasing vCores and see if that helps?
- Instead of taking huge data at a time, Could you please try re-running your data flow by splitting your source data to multiple iterations and see if that helps?
-
Utsav Chanda 71 Reputation points
2021-09-08T16:01:32.27+00:00 @ShaikMaheer-MSFT I agree with you that the error message was not at all helpful and that is why I rated it 1 star:)
Now coming to the issue, it was resolved after recreating the ACLs with Access ACL as well as Default ACL on the SINK folder structure. Not sure if the same would work for everyone who faces this issue in future -
ShaikMaheer-MSFT 38,401 Reputation points Microsoft Employee
2021-09-09T10:05:00.747+00:00 Hi @Utsav Chanda ,
Thank you for sharing your work around. Just curious to know on which thread you gave 1 star? Could you please help on it?
-
Utsav Chanda 71 Reputation points
2021-09-09T11:06:38.517+00:00 The Details of the Error Message. For error messages, MSFT prompts to rate the error message. That is where you put 1 star
-
ShaikMaheer-MSFT 38,401 Reputation points Microsoft Employee
2021-09-10T16:29:27.883+00:00 Hi @Utsav Chanda ,
Got you thank you for helping to understand it.
Sign in to comment