How to integrate/add more metrics & info into Ganglia UI in Databricks Jobs

Ramya Harinarthini_MSFT 5,331 Reputation points Microsoft Employee
2020-05-08T08:37:57.927+00:00

As per https://learn.microsoft.com/en-us/azure/databricks/clusters/clusters-manage#monitor-performance, Ganglia metrics Collection Period Snapshot modifications can be done using init scripts.

Could you please help with pointers to modify by default Ganglia dashboard provided in Databricks jobs.

[Note: As we migrate from MSDN, this question has been posted by an Azure Cloud Engineer as a frequently asked question]

Sourced from MSDN – Azure Databricks

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,072 questions
0 comments No comments
{count} votes

Accepted answer
  1. PRADEEPCHEEKATLA-MSFT 85,351 Reputation points Microsoft Employee
    2020-05-08T09:06:45.903+00:00

    Welcome to the Microsoft Q&A platform.

    Happy to answer your questions

    The pre-installed Ganglia UI is not modifiable in that way. If you want to create custom dashboards, you should set up your own monitoring system by using Azure Log Analytics.

    One option is to use Azure Monitor + Power BI dashboards: https://learn.microsoft.com/en-us/azure/architecture/databricks-monitoring/

    OR

    Monitoring and Logging in Azure Databricks With Azure Log Analytics and Grafana: https://cloudarchitected.com/2019/04/monitoring-azure-databricks/

    Hope this helps.

    Sourced from MSDN – Azure Databricks

    0 comments No comments

0 additional answers

Sort by: Most helpful