Data Flow failed with : Livy Id=[0] Job failed during run time with state=[dead].

Heta Desai 357 Reputation points
2023-03-15T13:29:27.56+00:00

Dataflow that pulls data from Azure sql database got failed with below error. I am executing the Dataflow inside a loop to pull multiple table from azure sql database. for few tables dataflow was running perfectly and after that it got failed.

Livy Id=[0] Job failed during run time with state=[dead].

I have attached a log for dataflow.

Azure-IR-11-application_1678861376088_0001-Driver (stderr) - stderr-2023-03-15-09.log

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,385 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator
    2023-03-28T00:11:39.69+00:00

    Hello Heta Desai,

    Thanks for the reply and providing the error details.

    After checking with my internal team, here is a possible cause and resolution for this issue.

    This error occurs when a pipeline is debugged with debug session and dies before the dataflow activity starts. For example, the pipeline has other activities before the dataflow run starts.

    In general, the debug session has a TTL of 1 hour.

    Can you please perform the below and debug your pipeline?

    Please increase the timeout of the debug session to 4 hours to keep the debug session alive for a longer time.

    If this still doesn't resolve your issue, please let me know your pipeline run ID and the synapse region for further troubleshooting.

    User's image


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.