Azure Databricks: "java.sql.SQLTransientConnectionException: elasticspark - Connection is not available, request timed out after 10000ms."

DisplayName42 56 Reputation points
2023-04-13T11:58:21.56+00:00

We have a Data Factory pipeline which runs Azure Databricks notebooks. This pipeline has been working for months without issues. In the last 2 weeks we got the following error message already a few times: Run result unavailable: job failed with error message java.sql.SQLTransientConnectionException: elasticspark - Connection is not available, request timed out after 10000ms. Cause: elasticspark - Connection is not available, request timed out after 10000ms. The pipeline starts every 2 hours and runs 13 notebooks in parallel. My questions:

  • Is there a way to get more information about this error? Why are there no logs on the databricks run page?
  • What does "elasticspark" mean in this error message?
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,973 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,755 questions
{count} vote