Expose spark metrics to prometheus

Long Tran 16 Reputation points

I want to expose spark cluster metrics in azure databrick to prometheus using Prometheus Serverlet. So I tried to edit the metrics.properties file to something like this


using the init script


cp /dbfs/databricks/metrics-conf/metrics.properties /databricks/spark/conf/

The file is successfully edited but I still cannot access the prometheus endpoint. As I checked the logs from spark, I don't see any prometheus servlet was created

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,955 questions
{count} vote