databricks performance

Question

what is the basic steps to improve azure databricks performance please help

Answer 1

Hi @Anshal ,

Thanks for using Microsoft Q&A forum and posting your query.

While considering Azure Databricks performance, you will have to review below stages to know what are the bottlenecks that needs an attention.

• Job and stage latency
• Sum of task execution latency
• Task metrics per stage
• Streaming throughput and latency
• Resource consumption per executor

For a detailed description of each of the above stages, please refer to this doc which has a clear explanation with images/samples: Troubleshoot performance bottlenecks in Azure Databricks

Hope this info helps.

----------

• Please don't forget to click on and upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators