Share via

Data Flow Failing: Meaningless Error Message

Will Page 26 Reputation points
2020-09-08T03:04:31.413+00:00

Hi,

I have a normally working data flow that has failed on a run with the following error:

java.lang.Exception: DataFlowManagerCLient.getPayload: 400

No matter how many times I re-run the data flow the same thing happens. All previous runs (for a long time) have been successful so I assume there's something wrong with the data in one of the sources.

The input files are CSV and they have passed a validation that they have have the correct column names.

I have switched on debug mode with the same set of input data and I can preview data without errors, so I guess the fault is beyond the limit of the debug row limit (set to 10000).

Looking at the past pipeline run in the Data flow details, I can see the above error message in red text, but there's no indication at what step in the data flow this is occurring. The feedback seems limited. Where can I find more verbose logs?

23152-annotation-2020-09-08-150346.png

Will.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.

3 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Keynes Yang 1 Reputation point Microsoft Employee
    2020-09-09T10:20:11.61+00:00

    There was a degraded service issue, impacting reliability and the engineering team fixed the problem on 02:30 UTC on 09 Sep 2020. Retries would have been helpful during that timeframe. Hope this helps.

    0 comments
  2. KranthiPakala-MSFT 46,747 Reputation points Microsoft Employee Moderator
    2020-09-09T20:56:02.767+00:00

    Thanks for the update @Keynes Yang .

    Hi @Will Page , as per confirmation from Keynes, there was a service issue around the time when you have experienced this behavior and the issue was fixed by ADF engineering team around 02:30 UTC on 09 Sep 2020.

    Could you please confirm if you're issue was resolved or it still persists? In-case if the issue still persists, please do share the runID of the pipeline so that we can work with engineering team for deeper analysis.

    Thank you for your patience and apologies for the inconvenience caused.

    Looking forward to your confirmation.

  3. Sandeep Gunda 1 Reputation point
    2021-08-09T16:10:30.28+00:00

    Hi Team,

    I am getting below Error.

    java.lang.Exception: DataFlowManagerClient.getPayload: 400 - RunId: 6065d5df-6a5c-4a00-8b6f-efad7117b20e

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.