Time to live

Question

what is time to live in databricks and how it help in reducing cost

Answer 1

Hello @Anshal ,

Thanks for the question and using MS Q&A platform.

Time to live: The allowed idle time for the data flow compute. Specifies how long it stays alive after completion of a data flow run if there are no other active jobs.

For example, if an activity run takes 6 minutes and timetolive is set to 5 minutes, the cluster stays alive for 5 minutes after the 6 minutes of processing the activity run. If another activity run is executed with the 6-minutes window, it is processed by the same cluster.

ADF has added a TTL (time-to-live) option to the Azure Integration Runtime for Data Flow properties to reduce data flow activity times.

The advantage of creating your own runtime, is you can choose a different cluster size (8 is the minimum) and a time to live smaller than one hour. Remember, if a cluster is paused, you don’t pay but you do when it’s running. This means you want to pause it as soon as possible to save costs.

For more details, refer to ADF adds TTL to Azure IR to reduce Data Flow activity times.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators