Error when deploying alerts on AKS metric: "node_cpu_usage_percentage"

Question

Hi everyone, we're currently facing an issue when creating alerts scoped on an Azure Kubernetes Service cluster, to monitor the node cpu usage percentage.

A couple days back we noticed the deploy broke both via Azure Pipelines / Terraform (self-made IaC) and via Azure Portal GUI. In both ways, we receive a 400 Bad Request error which says The metric with the name node_cpu_usage_percentage does not exist.

On the resource itself, Monitoring blade, Metrics do show node cpu usage percentage when chosen from the dropdown menu, and dashboard related to the metrics correctly shows data.

Payload from web page returns, among the other things (forget the YAML format, it's stuff printed in pipeline logs):

  "name": {
  "value": "node_cpu_usage_percentage",
  "localizedValue": "CPU Usage Percentage"
  },

This highly suggest the metric still exists :) i guess...

Again, the deploy worked previously both via IaC and GUI, and we made no changes to our pipelines.

Are we missing something?

Answer 1

After several days not working it just fixed itself. Whatevs.

Answer 2

The metric node_cpu_usage_percentage is valid for AKS, but metric alerts must be created against the correct metric namespace and API version for Microsoft.Insights/metricAlerts. A 400 error stating that the metric does not exist typically means the alert definition is not aligned with how Azure Monitor exposes that metric, even if the metric is visible in the Metrics blade.

Key points from the AKS monitoring reference:

• CPU Usage Percentage is exposed as:
  • Display name: CPU Usage Percentage
  • REST name: node_cpu_usage_percentage
  • Aggregations: Maximum, Average
  • Dimensions: node, nodepool
  • Time grains: PT1M, PT5M, PT15M, PT30M, PT1H, PT6H, PT12H

To get the alert working again, align the alert definition with how Azure Monitor expects the metric:

1. In the Azure portal, go to the AKS cluster → Monitoring → Metrics.
2. Select the CPU Usage Percentage metric and note:
   • The metric namespace shown in the UI.
   • The aggregation (for example, Average).
3. When defining the metric alert (via portal, Bicep, ARM, or Terraform using Microsoft.Insights/metricAlerts):
   • Set metricName to node_cpu_usage_percentage.
   • Set metricNamespace to the exact namespace shown in the Metrics blade for that metric.
   • Set timeAggregation to one of the supported values (for example, Average).
   • Ensure the API version used for Microsoft.Insights/metricAlerts is one of the documented versions (for example, 2018-03-01 or 2024-03-01-preview) and that the schema matches that version.

For Terraform/AzAPI definitions, the relevant properties are:

• metricName (required): must be node_cpu_usage_percentage.
• metricNamespace: must match the namespace used by the AKS metric.
• timeAggregation (required): one of Average, Maximum, Minimum, Total, or Count.
• criterionType: for static thresholds, use StaticThresholdCriterion.

If the alert previously worked and suddenly started failing, the most common causes are:

• The metric alert resource was being created without metricNamespace, and a platform change now requires it for AKS metrics.
• The metricAlerts API version or schema in the IaC was updated and no longer matches the way the metric is exposed.

Validating and explicitly setting metricNamespace, metricName, and timeAggregation in the alert criteria, using a supported metricAlerts API version, should resolve the 400 error.

---

References:

• Azure Kubernetes Service monitoring data reference
• Monitor Azure Kubernetes Service (AKS)
• Proactive monitoring best practices for Azure Kubernetes Service (AKS)
• Microsoft.Insights metricAlerts (deployment-language-terraform)
• Microsoft.Insights metricAlerts 2018-03-01 (deployment-language-terraform)
• Microsoft.Insights metricAlerts 2024-03-01-preview (deployment-language-terraform)