Collect Prometheus metrics from AKS cluster (preview)

This article describes how to configure your Azure Kubernetes Service (AKS) cluster to send data to Azure Monitor managed service for Prometheus. When you configure your AKS cluster to send data to Azure Monitor managed service for Prometheus, a containerized version of the Azure Monitor agent is installed with a metrics extension. You just need to specify the Azure Monitor workspace that the data should be sent to.

Note

The process described here doesn't enable Container insights on the cluster even though the Azure Monitor agent installed in this process is the same one used by Container insights. See Enable Container insights for different methods to enable Container insights on your cluster. See Collect Prometheus metrics with Container insights for details on adding Prometheus collection to a cluster that already has Container insights enabled.

Prerequisites

Enable Prometheus metric collection

Use any of the following methods to install the Azure Monitor agent on your AKS cluster and send Prometheus metrics to your Azure Monitor workspace.

  1. Open the Azure Monitor workspaces menu in the Azure portal and select your cluster.

  2. Select Managed Prometheus to display a list of AKS clusters.

  3. Click Configure next to the cluster you want to enable.

    Screenshot of Azure Monitor workspace with Prometheus configuration.

Verify Deployment

Run the following command to which verify that the daemon set was deployed properly:

kubectl get ds ama-metrics-node --namespace=kube-system

The output should resemble the following:

User@aksuser:~$ kubectl get ds ama-metrics-node --namespace=kube-system
NAME               DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
ama-metrics-node   1         1         1       1            1           <none>          10h

Run the following command to which verify that the replica set was deployed properly:

kubectl get rs --namespace=kube-system

The output should resemble the following:

User@aksuser:~$kubectl get rs --namespace=kube-system
NAME                            DESIRED   CURRENT   READY   AGE
ama-metrics-5c974985b8          1         1         1       11h
ama-metrics-ksm-5fcf8dffcd      1         1         1       11h

Limitations

  • CPU and Memory requests and limits can't be changed for Container insights metrics addon. If changed, they'll be reconciled and replaced by original values in a few seconds.
  • Metrics addon doesn't work on AKS clusters configured with HTTP proxy.

Uninstall metrics addon

Currently, Azure CLI is the only option to remove the metrics addon and stop sending Prometheus metrics to Azure Monitor managed service for Prometheus.

If you don't already have it, install the aks-preview extension with the following command.

The aks-preview extension needs to be installed using the following command. For more information on how to install a CLI extension, see Use and manage extensions with the Azure CLI.

az extension add --name aks-preview

Use the following command to remove the agent from the cluster nodes and delete the recording rules created for the data being collected from the cluster along with the Data Collection Rule Associations (DCRA) that link the DCE or DCR with your cluster. This doesn't remove the DCE, DCR, or the data already collected and stored in your Azure Monitor workspace.

az aks update --disable-azuremonitormetrics -n <cluster-name> -g <cluster-resource-group>

Region mappings

When you allow a default Azure Monitor workspace to be created when you install the metrics addon, it's created in the region listed in the following table.

AKS Cluster region Azure Monitor workspace region
australiacentral eastus
australiacentral2 eastus
australiaeast eastus
australiasoutheast eastus
brazilsouth eastus
canadacentral eastus
canadaeast eastus
centralus centralus
centralindia centralindia
eastasia westeurope
eastus eastus
eastus2 eastus2
francecentral westeurope
francesouth westeurope
japaneast eastus
japanwest eastus
koreacentral westeurope
koreasouth westeurope
northcentralus eastus
northeurope westeurope
southafricanorth westeurope
southafricawest westeurope
southcentralus eastus
southeastasia westeurope
southindia centralindia
uksouth westeurope
ukwest westeurope
westcentralus eastus
westeurope westeurope
westindia centralindia
westus westus
westus2 westus2
westus3 westus
norwayeast westeurope
norwaywest westeurope
switzerlandnorth westeurope
switzerlandwest westeurope
uaenorth westeurope
germanywestcentral westeurope
germanynorth westeurope
uaecentral westeurope
eastus2euap eastus2euap
centraluseuap westeurope
brazilsoutheast eastus
jioindiacentral centralindia
swedencentral westeurope
swedensouth westeurope
qatarcentral westeurope

Next steps