Edit

Configure remote write for Azure Monitor managed service for Prometheus using Microsoft Azure Active Directory pod identity (preview)

  • Article

Note

The remote write sidecar should only be configured via the following steps only if the AKS cluster already has the Azure AD pod enabled. This approach is not recommended as AAD pod identity has been deprecated to be replace by Azure Workload Identity

To configure remote write for Azure Monitor managed service for Prometheus using Azure AD pod identity, follow the steps below.

  1. Create user assigned identity or use an existing user assigned managed identity. For information on creating the managed identity, see Configure remote write for Azure Monitor managed service for Prometheus using managed identity authentication.

  2. Assign the Managed Identity Operator and Virtual Machine Contributor roles to the managed identity created/used in the previous step.

    az role assignment create --role "Managed Identity Operator" --assignee <managed identity clientID> --scope <NodeResourceGroupResourceId> 

az role assignment create --role "Virtual Machine Contributor" --assignee <managed identity clientID> --scope <Node ResourceGroup Id>

    The node resource group of the AKS cluster contains resources that you will require for other steps in this process. This resource group has the name MC_<AKS-RESOURCE-GROUP><AKS-CLUSTER-NAME><REGION>. You can locate it from the Resource groups menu in the Azure portal.

  3. Grant user-assigned managed identity Monitoring Metrics Publisher roles.

    az role assignment create --role "Monitoring Metrics Publisher" --assignee <managed identity clientID> --scope <NodeResourceGroupResourceId>

  4. Create AzureIdentityBinding

    The user assigned managed identity requires identity binding in order to be used as a pod identity. Run the following commands:

    Copy the following YAML to the aadpodidentitybinding.yaml file.

    
apiVersion: "aadpodidentity.k8s.io/v1" 

kind: AzureIdentityBinding 
metadata: 
 name: demo1-azure-identity-binding 
spec: 
 AzureIdentity: “<AzureIdentityName>” 
 Selector: “<AzureIdentityBindingSelector>”

    Run the following command:

    kubectl create -f aadpodidentitybinding.yaml

  5. Add a aadpodidbinding label to the Prometheus pod.
    The aadpodidbinding label must be added to the Prometheus pod for the pod identity to take effect. This can be achieved by updating the deployment.yaml or injecting labels while deploying the sidecar as mentioned in the next step.

  6. Deploy side car and configure remote write on the Prometheus server.

    1. Copy the YAML below and save to a file.
    prometheus: 
  prometheusSpec: 
    podMetadata: 
      labels: 
        aadpodidbinding: <AzureIdentityBindingSelector> 
    externalLabels: 
      cluster: <AKS-CLUSTER-NAME> 
    remoteWrite: 
    - url: 'http://localhost:8081/api/v1/write' 
    containers: 
    - name: prom-remotewrite 
      image: <CONTAINER-IMAGE-VERSION> 
      imagePullPolicy: Always 
      ports: 
        - name: rw-port 
      containerPort: 8081 
      livenessProbe: 
        httpGet: 
          path: /health
          port: rw-port
          initialDelaySeconds: 10 
          timeoutSeconds: 10 
      readinessProbe: 
         httpGet: 
          path: /ready
          port: rw-port
          initialDelaySeconds: 10 
          timeoutSeconds: 10 
    env: 
      - name: INGESTION_URL 
        value: <INGESTION_URL> 
      - name: LISTENING_PORT 
        value: '8081' 
      - name: IDENTITY_TYPE 
        value: userAssigned 
      - name: AZURE_CLIENT_ID 
        value: <MANAGED-IDENTITY-CLIENT-ID> 
      # Optional parameter 
      - name: CLUSTER 
        value: <CLUSTER-NAME>

    b. Use helm to apply the YAML file to update your Prometheus configuration with the following CLI commands.

    # set context to your cluster 
az aks get-credentials -g <aks-rg-name> -n <aks-cluster-name> 
# use helm to update your remote write config 
helm upgrade -f <YAML-FILENAME>.yml prometheus prometheus-community/kube-prometheus-stack --namespace <namespace where Prometheus pod resides>

Next steps