Caused by: com.databricks.NotebookExecutionException: FAILED

Question

We have Databricks pipeline build and running using datafactory in batches with multiple files. At random, processing of a given file will fail causing the automation to abort processing that file and move on to the next. The error is consistent but indescript: “Caused by: com.databricks.NotebookExecutionException: FAILED” and the troubleshooting documentation for the error code thrown (3204) says “N/A” and applies to issues such as “an unexpected cluster state”. When we rerun the files through the same orchestration and pipelines or manually they typically work fine

Do you have any recommendations for troubleshooting this issue? Trace from DF.

DataBricks Stack Trace from DataFactory Output URL:
at com.databricks.workflow.WorkflowDriver.run(WorkflowDriver.scala:75)
at com.databricks.dbutils_v1.impl.NotebookUtilsImpl.run(NotebookUtilsImpl.scala:122)
at line1ee345426a1546219668c06720be0c2627.$read$$iw$$iw$$iw$$iw$$iw$$iw.\

Answer 1

Hello @Jeeva ,

I would recommend you to enable retry in the activity using the below option and check if that works.

Hope this helps. Do let us know if you any further queries.

------------

• Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
• Want a reminder to come back and check responses? Here is how to subscribe to a notification.

Answer 2

Hi,
We do exact the same ETL: using Azure Data Factory to orchestrate Python Databricks notebooks and we experience the same behavior.

The retry policy solves the issue, but is there no better way to solve this?