Data Flow failed with : Livy Id=[0] Job failed during run time with state=[dead].

heta desai 247 Reputation points
2023-03-15T13:29:27.56+00:00

Dataflow that pulls data from Azure sql database got failed with below error. I am executing the Dataflow inside a loop to pull multiple table from azure sql database. for few tables dataflow was running perfectly and after that it got failed.

Livy Id=[0] Job failed during run time with state=[dead].

I have attached a log for dataflow.

Azure-IR-11-application_1678861376088_0001-Driver (stderr) - stderr-2023-03-15-09.log

Azure SQL Database
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,402 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. BhargavaGunnam-MSFT 26,316 Reputation points Microsoft Employee
    2023-03-28T00:11:39.69+00:00

    Hello Heta Desai,

    Thanks for the reply and providing the error details.

    After checking with my internal team, here is a possible cause and resolution for this issue.

    This error occurs when a pipeline is debugged with debug session and dies before the dataflow activity starts. For example, the pipeline has other activities before the dataflow run starts.

    In general, the debug session has a TTL of 1 hour.

    Can you please perform the below and debug your pipeline?

    Please increase the timeout of the debug session to 4 hours to keep the debug session alive for a longer time.

    If this still doesn't resolve your issue, please let me know your pipeline run ID and the synapse region for further troubleshooting.

    User's image