Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
7,586 questions
Is there any fix provided for high memory usage ?
High memory consumption with AKS v1.25.2
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 68%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ET%3C/text%3E%3C/svg%3E)
Tanul
1,251
Reputation points
Team,
Since the day I have updated the AKS to v1.25.2, I can see huge spikes and node memory pressure issues.
Nodes are always consuming 135 to 140% of memory.. Till the time I was at 1.24.9 everything was working fine.
Just now, I saw that port.azure.com has removed the v1.25.2 version from Create new-->Azure kubernetes cluster section. Does this version of AKS has any problem. Should we immediately switch to any other version for resolving memory issue.
Problem areas
- These 2 pods out of 9 of them
- ama-logs
- ama-logs-rs Takes more that 400 Mi of memory.. Its very difficult to accommodate them in B2S nodes.
- These 16 pods(8 each)
- csi-azuredisk-node
- csi-azurefile-node Takes 910 Mi of memory. I even raised the support ticket but customer support was unable to figure out whether we are using them or not. In addition, unable to suggest that when we should keep or why we should keep.
- I have also observed that AKS 1.24.x version had
ubuntu 18but AKS 1.25.x version hasubuntu 22.
Could you please suggest some resolution. Thank you.
Kind Regards,
Tanul
My AKS Configuration:- 8 nodes of Standard B2s size as its a non-prod environment.
Pod structure:- Below are the listed pods and their memory consumption except the default microsoft pods(which are taking 4705 Mi of memory in total) running inside cluster
- Dameon set of AAD pod identity:- Taking total 191 Mi of memory
- Total 2 pods of kong :- Taking total 914 Mi Memory
- Daemon set of twistlock vulnerability scanner:- Taking total 1276 Mi of memory
- Total 10 pods of our .net microservices:- Taking total 820 Mi of memory
{count} vote
-
Tanul 1,251 Reputation points
2023-02-03T07:26:16.3566667+00:00 Hello Microsoft,
Any suggestions.. Its seems to be a global problem with 1.25.x version.. Check this thread. Any suggestion please.
https://github.com/Azure/AKS/issues/3443#issuecomment-1415196080
-
Tanul 1,251 Reputation points
2023-02-03T09:17:14.1133333+00:00 @KarishmaTiwari-MSFT , Hey karishma.. Is it possible to raise the support ticket with your help..
We have raised the support ticket few days ago and support team conveyed that they don't have hands-on experience with kubernetes and they have started working on it few months back.
Would be grateful if your team can help us here as the engineering teams are facing lot of challenges. If its ok with your team, could you please share the procedure to send a mail for support ticket on this. Thank you.
Kind Regards,
Tanul -
Tanul 1,251 Reputation points
2023-02-03T17:05:46.39+00:00 @vipullag-MSFT , Hello vipul, could you please guide us here. Thank you.
-
Tanul 1,251 Reputation points
2023-02-06T09:43:26.6433333+00:00 @prmanhas-MSFT , Hey preeti, Trust you are doing well.. Actually we are facing some challenge related to performance.. If possible could you please guide us a little.. Our engineers are facing lot of issues.. Thank you.
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 35%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
srbhatta-MSFT 8,551 Reputation points Microsoft Employee2023-02-08T13:57:20.7333333+00:00 Hello @Tanul ,
Thanks for reaching out to us. Could you please reach out to the Azure Support Team by logging a support ticket with them and they can assist you on this issue, as this requires analysis of logs and troubleshooting. If you do not have a support plan enabled, then I can enable a free support request for you. You can drop an email to AzCommunity@microsoft.com with the subject line ATTN : srbhatta , and please mention your subscription id on the email, and I will reach back to you with the next steps. Thank you.
Sign in to comment
1 answer
Sort by: Most helpful
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 84%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Saurabh Bhargava 1 Reputation point Microsoft Employee2023-08-29T06:45:04.3+00:00