Is it possible to define Auto-termination activity in Azure Databricks Jobs cluster

Question

Hello Team,

While creating azure databricks job cluster I figured out that auto-termination is disabled.

Requirement is to keep job cluster active for 5 mins even when the job fails. So is there any possible way to achieve it?

Note: Interactive cluster is not used in this case. Above requirement is to only achieve with job cluster if it is possible.

Regards,

Ashwini G

Answer 1

@ASHWINI GAIKWAD - Thanks for the question and using MS Q&A platform.

When you run a job on a New Job Cluster (which is usually recommended), the cluster terminates and is unavailable for restarting when the job is complete. On the other hand, if you schedule a job to run on an Existing All-Purpose Cluster that has been terminated, that cluster will autostart.

For more details, refer to Azure Databricks - Terminate a cluster and Auto termination is disabled when starting a job cluster.

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.