Certificate rotation in Azure Kubernetes Service (AKS)

Azure Kubernetes Service (AKS) uses certificates for authentication with many of its components. You need to periodically rotate those certificates for security or policy reasons. This article shows you how certificate rotation works in your AKS cluster.

Prerequisites

This article requires the Azure CLI version 2.0.77 or later. Check your version using the az --version command. If you need to install or upgrade, see Install Azure CLI.
Configure kubectl to connect to your AKS cluster using the az aks get-credentials command:
```
az aks get-credentials --resource-group <resource-group> --name <cluster-name>
```

AKS certificates, Certificate Authorities, and Service Accounts

AKS generates and uses the following certificates, Certificate Authorities (CA), and Service Accounts (SA):

The AKS API server creates a CA called the Cluster CA, which signs certificates for one-way communication from the API server to kubelet.
Each kubelet creates a Certificate Signing Request (CSR), which the Cluster CA signs, for communication from the kubelet to the API server.
The API aggregator uses the Cluster CA to issue certificates for communication with other APIs. The API aggregator can also have its own CA for issuing those certificates, but it currently uses the Cluster CA.
Each agent node uses an SA token, which the Cluster CA signs.
The kubectl client has a certificate for communicating with the AKS cluster.

Microsoft maintains all certificates mentioned in this section, except for the cluster certificate.

Certificate expiration dates

Important

The expiration date for your certificates depends on when your AKS cluster was created:

AKS clusters created before May 2019 have certificates that expire after two years.
AKS clusters created after May 2019 have Cluster CA certificates that expire after 30 years.

You can verify when your cluster was created using the kubectl get nodes command, which shows you the Age of your agent nodes.

Check cluster certificate expiration date

Check the expiration date of the cluster certificate using the kubectl config view command.

kubectl config view --raw -o jsonpath="{.clusters[?(@.name == '')].cluster.certificate-authority-data}" | base64 -d | openssl x509 -text | grep -A2 Validity

Check API server certificate expiration date

Check the expiration date of the API server certificate using the following curl command:
```
curl https://{apiserver-fqdn} -k -v 2>&1 | grep expire
```

Check virtual machine (VM) agent node certificate expiration date

Check the expiration date of the VM agent node certificate using the az vm run-command invoke command.

Key parameters in this command: - --resource-group <node-resource-group>: The resource group that contains the VM agent node. - --name <vm-name>: The name of the VM agent node. - --scripts "openssl x509 -in /etc/kubernetes/certs/apiserver.crt -noout -enddate": The script that retrieves the expiration date of the API server certificate located at /etc/kubernetes/certs/apiserver.crt.
```
az vm run-command invoke --resource-group <node-resource-group> --name <vm-name> --command-id RunShellScript --query 'value[0].message' -otsv --scripts "openssl x509 -in /etc/kubernetes/certs/apiserver.crt -noout -enddate"
```

Check certificate expiration for the Azure Virtual Machine Scale Set agent node

Check the expiration date of the Azure Virtual Machine Scale Set agent node certificate using the az vmss run-command invoke command.

Key parameters in this command: - --resource-group <node-resource-group>: The resource group that contains the Azure Virtual Machine Scale Set agent node. - --name <vmss-name>: The name of the Azure Virtual Machine Scale Set. - --instance-id 1: The instance ID of the Azure Virtual Machine Scale Set agent node. - --scripts "openssl x509 -in /var/lib/kubelet/pki/kubelet-client-current.pem -noout -enddate": The script that retrieves the expiration date of the kubelet client certificate located at /var/lib/kubelet/pki/kubelet-client-current.pem.
```
az vmss run-command invoke --resource-group <node-resource-group> --name <vmss-name> --command-id RunShellScript --instance-id 1 --scripts "openssl x509 -in  /var/lib/kubelet/pki/kubelet-client-current.pem -noout -enddate" --query "value[0].message"
```

Manually rotate your cluster certificates

Rotate all certificates, CAs, and SAs on your cluster using the az aks rotate-certs command.
```
az aks rotate-certs --resource-group <resource-group> --name <cluster-name>
```
Important

The az aks rotate-certs command recreates all of your agent nodes, Azure Virtual Machine Scale Sets, and disks. This command can also cause up to 30 minutes of downtime for your AKS cluster. If the command fails before completing, use the [az aks show][az-aks-show] command to verify the status of the cluster is Certificate Rotating. If the cluster is in a failed state, rerun the az aks rotate-certs command to rotate your certificates again.
Verify the old certificates are no longer valid using any kubectl command. The following example uses the kubectl get nodes command:
```
kubectl get nodes
```
If you didn't update the certificates used by kubectl, you see an error similar to the following example output:
```
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "ca")
```
Update the certificate used by kubectl using the az aks get-credentials command with the --overwrite-existing flag.
```
az aks get-credentials --resource-group <resource-group> --name <cluster-name> --overwrite-existing
```
Verify the certificates are updated using the kubectl get command.
```
kubectl get nodes
```

If you have any services that run on top of AKS, you might need to update their certificates as well.

Rotate the kubelet serving certificate

When you rotate the kubelet serving certificate, AKS allows kubelet server Transport Layer Security (TLS) Bootstrapping for both bootstrapping and rotating serving certificates signed by the Cluster CA.

Limitations for kubelet serving certificate rotation

Supported on Kubernetes version 1.27 and above.
Not supported when the node pool is using a node pool snapshot based on any node image older than 202501.12.0.
You can't manually enable this feature. Kubelet serving certificate rotation is enabled by default on existing node pools when they perform their first upgrade to any Kubernetes version 1.27 or higher. Kubelet serving certificate rotation is enabled by default on new node pools using Kubernetes version 1.27 or higher. To see if kubelet serving certificate rotation is enabled in your region, check the AKS releases.

Verify kubelet serving certificate rotation is enabled

Each node with the feature enabled is automatically given the label kubernetes.azure.com/kubelet-serving-ca=cluster.

Verify the labels are set using the kubectl get nodes -L kubernetes.azure.com/kubelet-serving-ca command.
```
kubectl get nodes -L kubernetes.azure.com/kubelet-serving-ca
```
The output should show the label kubernetes.azure.com/kubelet-serving-ca with the value cluster for each agent node.

Verify kubelet TLS Bootstrapping is working

Verify the bootstrapping process is taking place using the kubectl get command.

kubectl get csr --field-selector=spec.signerName=kubernetes.io/kubelet-serving

In the output, all serving CSRs should be in the Approved,Issued state, which indicates the CSR was approved and issued a signed certificate. Serving CSRs have a signer name of kubernetes.io/kubelet-serving. For example:

NAME        AGE    SIGNERNAME                                    REQUESTOR                    REQUESTEDDURATION   CONDITION
csr-1ab2c   113s   kubernetes.io/kube-apiserver-client-kubelet   system:bootstrap:uoxr9r      none              Approved,Issued
csr-defgh   111s   kubernetes.io/kubelet-serving                 system:node:akswinp7000000   none              Approved,Issued
csr-ij3kl   46m    kubernetes.io/kubelet-serving                 system:node:akswinp6000000   none              Approved,Issued
csr-mn4op   46m    kubernetes.io/kube-apiserver-client-kubelet   system:bootstrap:ho7zyu      none              Approved,Issued

Verify kubelet is using a certificate obtained from server TLS Bootstrapping

Confirm the kubelet is using a serving certificate signed by the Cluster CA using the kubectl debug command.
```
kubectl debug node/<node> -ti --image=mcr.microsoft.com/azurelinux/base/core:3.0 -- ls -l /host/var/lib/kubelet/kubelet-server-current.pem
```
If a kubelet-server-current.pem symlink exists, then the kubelet bootstrapped/rotated its own serving certificate, and the Cluster CA signed it.

Disable kubelet serving certificate rotation

Disable kubelet serving certificate rotation by updating the node pool using the az aks nodepool update command with the aks-disable-kubelet-serving-certificate-rotation=true tag.
```
az aks nodepool update --cluster-name <cluster-name> --resource-group <resource-group> --name <node-pool-name> --tags aks-disable-kubelet-serving-certificate-rotation=true
```

Reimage your nodes using a node image upgrade or by scaling the pool to zero instances and then back up to the desired value.

Certificate autorotation

Keep the following considerations in mind when using certificate autorotation:

If you have an existing cluster, you have to upgrade that cluster to enable certificate autorotation.
Don't disable TLS Bootstrap to keep certificate autorotation enabled.
If the cluster is in a stopped state during certificate autorotation, only the control plane certificates are rotated. In this case, you should recreate the node pool after certificate rotation to initiate the node pool certificate rotation.
For any AKS clusters created or upgraded after March 2022, AKS automatically rotates non-CA certificates on both the control plane and agent nodes within 80% of the client certificate valid time before they expire with no downtime for the cluster.

Verify TLS Bootstrapping is enabled on current agent node pool

Verify your cluster has TLS Bootstrapping enabled by browsing to one to the following paths:
- On a Linux node: /var/lib/kubelet/bootstrap-kubeconfig or /host/var/lib/kubelet/bootstrap-kubeconfig
- On a Windows node: C:\k\bootstrap-config
For more information, see Connect to Azure Kubernetes Service (AKS) cluster nodes for maintenance or troubleshooting.

Note

The file path might change as Kubernetes versions evolve.
Once a region is configured, create a new cluster or upgrade an existing cluster to set certificate autorotation for the cluster certificate. You need to upgrade the control plane and node pool to enable this feature.

Best practices for cluster security and upgrades in Azure Kubernetes Service (AKS).

Feedback

War dës Säit hëllefräich?

Last updated on 2025-12-17