How to patch and schedule Azure kubernestes nodes on VMSS

Question

How to patch and schedule Azure kubernestes nodes on VMSS

Azure-learning 56

what is the recommended way to patch Azure kubernetes nodes on VMSS or is it something Microsoft manage from their end for OS update .

if not what is the recommended way of doing and planning the scheduled patches on these aks nodes(vmss)

deherman-MSFT 38,021 Reputation points Microsoft Employee Moderator

2023-01-26T19:05:24.48+00:00

Hello,

I hope this has been helpful! We appreciate hearing from you and would love to help others who may have the same question. Accepting answers helps increase visibility of this question for other members of the Microsoft Q&A community. Thank you for helping to improve Microsoft Q&A!
Roy Kim (Azure MVP) 191 Reputation points MVP

2023-05-19T17:54:38.98+00:00

To patch the nodes you have to go the AKS resource and clock on the Node pool and update the node images.

Here is my 2-minute video that explains how to do it:
https://learn.microsoft.com/en-us/shows/one-dev-minute/how-do-you-patch-and-schedule-azure-kubernetes-nodes-on-vmss

1 answer

Your answer

deherman-MSFT 38,021 Reputation points Microsoft Employee Moderator

2023-01-26T19:05:24.48+00:00

Hello,

I hope this has been helpful! We appreciate hearing from you and would love to help others who may have the same question. Accepting answers helps increase visibility of this question for other members of the Microsoft Q&A community. Thank you for helping to improve Microsoft Q&A!
Roy Kim (Azure MVP) 191 Reputation points MVP

2023-05-19T17:54:38.98+00:00

To patch the nodes you have to go the AKS resource and clock on the Node pool and update the node images.

Here is my 2-minute video that explains how to do it:
https://learn.microsoft.com/en-us/shows/one-dev-minute/how-do-you-patch-and-schedule-azure-kubernetes-nodes-on-vmss

Answer 1

Andrei Barbu 2,596 Microsoft Employee

Hello @Azure-learning !

Thank you for raising the question.

In Azure Kubernetes Service (AKS), nodes of the same configuration are grouped together into node pools. These node pools contain the underlying VM/VMSS.

Please be informed that the OS (Linux or Windows) patches are included in the node image used by the AKS node (also called VM or instance). AKS provides one new image almost weekly with the latest updates. That is to address vulnerabilities, bring new features, fix issues etc. The node images can be tracked here:
https://github.com/Azure/AKS/tree/master/vhd-notes

You can patch/upgrade the node image of a node pool referring to the below command:

az aks nodepool upgrade \
--resource-group myResourceGroup \
--cluster-name myAKSCluster \
--name mynodepool \
--node-image-only

or to all the node pools (if you have more) with:

az aks upgrade \
--resource-group myResourceGroup \
--name myAKSCluster \
--node-image-only

Reference link: https://learn.microsoft.com/en-us/azure/aks/node-image-upgrade

If you would like to automate this, you can use auto-upgrade feature with node-image channel. Auto-upgrade works with planned maintenance feature if you'd like the upgrades to happen only at a specific time.

Please note that this will always bring you the latest node image version and you cannot choose a specific version.

Additionally, it is important to know that the default distribution used in AKS is Ubuntu. You can also go for Mariner, please refer to the following for that: https://learn.microsoft.com/EN-us/azure/aks/use-mariner

Until Kubernetes version 1.24 (including), Ubuntu 18.04 is used. From Kubernetes version 1.24, Ubuntu 22.04 will be used. More information: https://azure.microsoft.com/en-us/updates/generally-available-aks-support-for-ubuntu-2204/

I hope this answers your question.

If the ANSWER is helpful, please click "Accept Answer" and upvote it.

Thank you!

Azure-learning 56 Reputation points

2023-01-09T08:15:53.887+00:00

@Andrei Barbu Thank you for the suggestion.

Could you please let me know can we use Azure update management or Maintenance Configurations feature to patch the AKS nodes(VMSS) and schedule it periodically like we do for VMS
Andrei Barbu 2,596 Reputation points Microsoft Employee

2023-01-09T08:24:50.277+00:00

Hello @Azure-learning !

Azure update management or Maintenance Configurations feature are not directly supported to AKS because the VMSS part of AKS cluster are managed by AKS and actions over them should be performed only via AKS API, as per https://learn.microsoft.com/en-us/azure/aks/support-policies#user-customization-of-agent-nodes.

Fortunately, if you would like to automate upgrades in AKS, you can use auto-upgrade feature The available upgrade channels are:

node-image channel seems he one you are looking for, to upgrade the Ubuntu version of the nodes.

I would recommend you to look into https://learn.microsoft.com/en-us/azure/aks/auto-upgrade-cluster for more information.

Auto-upgrade works with planned maintenance feature if you'd like the upgrades to happen only at a specific time.

Reference link: https://learn.microsoft.com/en-us/azure/aks/planned-maintenance

If the ANSWER is helpful, please click "Accept Answer" and upvote it.

Thank you!
Azure-learning 56 Reputation points

2023-01-09T14:13:54.527+00:00

@Andrei Barbu Thank you ,I am going through the suggested link. also In the production system , will there be any specific strategy we need to consider , like rolling updates .
what is the recommended ways to do the node image upgrade in Production system?
Andrei Barbu 2,596 Reputation points Microsoft Employee

2023-01-09T14:51:46.817+00:00

@Azure-learning there is no specific guidance, but please note that during the node image upgrade operation (as well as for Kubernetes version upgrade operation), a buffer node is being added, then each node is being removed and new nodes with the new version are being added, one by one. So you may consider performing this kind of maintenance during a maintenance window.

An approach that I would suggest is having multiple node pools that would be upgraded one by one. While you upgrade one of them, you can move the application workload to the other and so on.

I hope you will find this helpful.

If the ANSWER is helpful, please click "Accept Answer" and upvote it.

Thank you!
Azure-learning 56 Reputation points

2023-01-10T10:02:21.103+00:00

@Andrei Barbu Thank you for the explanation . Sorry but got confused with the lines "While you upgrade one of them, you can move the application workload to the other and so on." , I believe , this will be done automatically when we start the upgrade command ,
Andrei Barbu 2,596 Reputation points Microsoft Employee

2023-01-10T10:05:38.13+00:00

@Azure-learning your understanding is correct. When I said "While you upgrade one of them, you can move the application workload to the other and so on." I was referring when upgrade one node pools, that you can drain the nodes part of the node pool that is going to be upgraded so you have more control. But during the upgrade operation, that also happens automatically.
Azure-learning 56 Reputation points

2023-01-10T11:12:34.147+00:00

Hi @Andrei Barbu .Thank you. sorry i have few more queries My AKS clusterversion is 1.22.15. as as per Micosoft Documentation "AKS is retiring v1.22.x on this (December 4th) release. Please upgrade your clusters to v1.23 or (preferably) above."

1 - In the portal , it's giving option to upgrade to v1.23.8 and v1.23.12 for cluster , Can we upgrade directly to v1.23.12 and skip v1.23.8? Since it's retired version , will there be any impact

2 -also the main requirement is to update nodeimage? Can i directly upgrade nodeimage ignoring point1 with nodepool upgrade -- option . or control plane also we need to upgrade , overall az aks upgrade is required ,not just nodeimage?

Current version AKSUbuntu-1804gen2containerd-2022.09.22 Latest version AKSUbuntu-1804gen2containerd-2022.12.19

3 - I have 3 node pools with 5 vmss in eachpool. we need to do it one by one in eachnode pool ?
Andrei Barbu 2,596 Reputation points Microsoft Employee

2023-01-10T11:48:31.193+00:00

@Azure-learning 1 - Yes, you can go directly to 1.23.12. Regarding the impact, there shouldn't be any. This is dependent of your workload, but generally speaking, there is no impact, newer versions bring new features, bug fixes etc.

2 - Yes, you can only upgrade the node image. You can do that for each node pool from portal by selecting it and clicking "Update image" or for all via az cli:

Each node pool:

az aks nodepool upgrade
--resource-group myResourceGroup
--cluster-name myAKSCluster
--name mynodepool
--node-image-only

or to all the node pools (if you have more) with:

az aks upgrade
--resource-group myResourceGroup
--name myAKSCluster
--node-image-only

Reference link: https://learn.microsoft.com/en-us/azure/aks/node-image-upgrade

But I would recommend you to upgrade the Kubernetes version as well. Additionally, if you perform a Kubernetes version upgrade, you will also get the latest AKSUbuntu-1804 version. Vice-versa is not applicable, if you upgrade node image, you will not get the Kubernetes version upgraded.

3 - You can upgrade all at once or upgrade the control plane and then one by one the node pools. It is totally fine to upgrade all of them at once. I mentioned that you can also upgrade one by one the node pools just to provide you more flexibility.

I hope you will find this helpful.

If the ANSWER is helpful, please click "Accept Answer" and upvote it.

Thank you!
Azure-learning 56 Reputation points

2023-01-10T13:58:29.147+00:00

Thank you @Andrei Barbu for your help. Thank you once again
Andrei Barbu 2,596 Reputation points Microsoft Employee

2023-01-10T15:23:35.933+00:00

Welcome @Azure-learning !

If the ANSWER I provided was helpful, if you can click "Accept Answer" and upvote it so other users will benefit from it.

Thank you for your cooperation!

Share via

How to patch and schedule Azure kubernestes nodes on VMSS

1 answer

Your answer