Data Flow Execution Fail state=[dead]

Question

Hi!

Since last Sunday, several pipelines are failing at the data flow activities with the following error message:

Yesterday I have tried to create a separate AutoResolve Integration Runtime but this does not fix it completely. I get the error with scheduled triggers, manual triggers and debug sessions. The error appears on data flow activities sinking one file and sinking multiple files. We also get successful completions on all the scenarios described above. There were no changes on the activities before this happened.

The livy logs shows this

24/07/16 06:03:34 INFO ResourceUtils: Unable to find 'resource-types.xml'.
24/07/16 06:03:34 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (14336 MB per container)
24/07/16 06:03:34 INFO Client: Will allocate AM container, with 7680 MB memory including 512 MB overhead
24/07/16 06:03:34 INFO Client: Setting up container launch context for our AM
24/07/16 06:03:34 INFO Client: Setting up the launch environment for our AM container
24/07/16 06:03:34 INFO Client: Preparing resources for our AM container
24/07/16 06:03:34 INFO Client: Uploading resource wasbs://synapsedataflowjars@adfdataflowweugen2.blob.core.windows.net/spark331/runmanager-synapse-spark331.jar -> xxxxx.jar
stderr: 
YARN Diagnostics: 
org.apache.livy.utils.submit.exceptions.SparkSubmitProcessTimedOutException: No YARN application is found with tag livy-batch-0-0qizz2tm in 300 seconds. This may be because 1) spark-submit fail to submit application to YARN; or 2) YARN cluster doesn't have enough resources to start the application in time. Please check Livy log and YARN log to know the details.

Thanks!

Answer 1

Hi @Rob Gouw | HBM Machines

Thanks for the question and using MS Q&A platform.

The product team deployed the fix, and the issue has been resolved. You should not see these errors anymore.

In case you still encounter any issues, please let me know.

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.