How can I expose and scrape AKS coredns autoscaler metrics to Prometheus?

Question

Hey,

I want to add a Grafana Kubernetes Cluster Autoscaler. To achieve that, I have to expose the autoscaler's metrics and add a way to scrape it - usually done with a ServiceMonitor.

Today this behavior is supported by coredns by settings prometheus.service.enabled=true and prometheus.monitor.enabled=true.

It seems that by default, AKS engine does not support this feature. AKS uses a raw YAML to install coredns-autoscaler instead of installing its helm chart.

Is there a way to achieve that without installing the coredns chart myself? I don't want to maintain the entire coredns stack, but I would be happy to add the autoscaler number to my Prometheus instance (I don't want to use Azure insights/monitoring).

Thank you

Answer 1

Hi @Shaked Klein Orbach ,

Thank you for posting your question.

The issue you are mentioning has been reported a while back, and is under works. Basically, Coredns pods are created with the annotation prometheus.io/port but the annotation prometheus.io/scrape is missing. That makes it hard to automatically scrape its metrics with an external Prometheus service.

To add the annotation to enable scraping, please run the below:
create a patcher file (coredns-patcher.yaml):

spec:  
  template:  
    metadata:  
      annotations:  
        prometheus.io/port: "9153"  
        prometheus.io/scrape: "true"  

then

kubectl patch -n kube-system deploy coredns --patch "$(cat coredns-patcher.yaml)"  

References:
AKS CoreDNS + Azure Prometheus - can't see metrics in log analytics
Missing prometheus.io/scrape annotation in coredns (this issue is still active)

Please "Accept as Answer" if it helped, so that it can help others in the community looking for help on similar topics.