Edit

Share via

Increased memory usage reported in Kubernetes 1.25 or later versions

  • Article

This article discusses how to resolve an increased reported memory usage issue in Microsoft Azure Kubernetes 1.25 or a later version.

Symptoms

You experience one or more of the following symptoms:

  • Pods report increased memory usage after you upgrade a Microsoft Azure Kubernetes Service (AKS) cluster to Kubernetes 1.25 or a later version.

  • A node reports memory usage that's greater than in earlier versions of Kubernetes when you run the kubectl top node command.

  • Increased pod evictions and memory pressure occur within a node.

Cause

This increase is caused by a change in memory accounting within version 2 of the Linux control group (cgroup) API. Cgroup v2 is now the default cgroup version for Kubernetes 1.25 on AKS.

Note

This issue is distinct from the memory saturation in nodes that's caused by applications or frameworks that aren't aware of cgroup v2. For more information, see Memory saturation occurs in pods after cluster upgrade to Kubernetes 1.25.

Solution

  • If you observe frequent memory pressure on the nodes, upgrade your subscription to increase the amount of memory that's available to your virtual machines (VMs).

  • If you see a higher eviction rate on the pods, use higher limits and requests for pods.

Note

If you experience only an increase in memory use without any of the other symptoms that are mentioned in the "Symptoms" section, you don't have to take any action.

Status

We're actively working with the Kubernetes community to fix the underlying issue, and we'll keep you updated on our progress. We also plan to change the eviction thresholds or resource reservations, depending on the outcome of the fix.

Reference

Third-party information disclaimer

The third-party products that this article discusses are manufactured by companies that are independent of Microsoft. Microsoft makes no warranty, implied or otherwise, about the performance or reliability of these products.

Third-party contact disclaimer

Microsoft provides third-party contact information to help you find additional information about this topic. This contact information may change without notice. Microsoft does not guarantee the accuracy of third-party contact information.

Contact us for help

If you have questions or need help, create a support request, or ask Azure community support. You can also submit product feedback to Azure feedback community.