Edit

Automate upgrades of Kubernetes and node images across multiple clusters using Azure Kubernetes Fleet Manager

Applies to: ✔️ Fleet Manager ✔️ Fleet Manager with hub cluster

Keeping clusters updated in a timely and safe fashion is a key concern of platform administrators. When you adopt Azure Kubernetes Fleet Manager Update Runs and Strategies, you can use auto-upgrade profiles to automate the execution of update runs when new Kubernetes or node image versions are released.

This article explains how to use auto-upgrade profiles to automatically create and execute update runs when Azure Kubernetes Service (AKS) releases new Kubernetes or node image versions.

Note

Auto-upgrade triggered update runs honor planned maintenance windows that you set at the AKS cluster level. For more information, see planned maintenance across multiple member clusters that learn how update runs handle member clusters with configured planned maintenance windows.

Before you begin

  • Read the conceptual overview of auto-upgrade profiles, which explains the configurations referenced in this guide.
  • Set up a Fleet Manager with one or more member clusters. If you don't have one, follow the quickstart to create a Fleet Manager and join AKS clusters as members.
  • Configure an update strategy. Use the instructions in the update run how-to article. You need the update strategy resource identifier to use with an auto-upgrade profile when using the Azure CLI.

  • Set the following environment variables to use in the Azure CLI commands:

    export GROUP=<resource-group>
export FLEET=<fleet-name>
export AUTOUPGRADEPROFILE=<upgrade-profile-name>
# Optional
export STRATEGYID=<strategy-id>
export CLUSTER=<aks-cluster-name>

  • Install latest Azure CLI version. To install or upgrade, see Install the Azure CLI.

  • Install the latest fleet Azure CLI extension. Use the az extension add command to install the extension.

    az extension add --name fleet

    Use the az extension update command to update to the latest version of the extension.

    az extension update --name fleet

  • Install and configure Terraform.

  • Install latest Azure CLI version. To install or upgrade, see Install the Azure CLI. You use the Azure CLI to verify the auto-upgrade profile that Terraform creates.

  • Install the latest fleet Azure CLI extension. Use the az extension add command to install the extension.

    az extension add --name fleet

    Use the az extension update command to update to the latest version of the extension.

    az extension update --name fleet

  • The Terraform sample used in this article creates its own resource group and Azure Kubernetes Fleet Manager, and it doesn't join any AKS clusters as Fleet members. To exercise the resulting auto-upgrade profile against real update runs, join existing AKS clusters as members of the Fleet after you apply the sample. For more information, see the quickstart.

Note

Clusters with agent pools created from node pool snapshots are affected as follows based on the selected auto-upgrade channel and node image option:

  • Node image channel: The node image upgrades to the version determined by Fleet Manager. The reference to the snapshot (creationData) is removed from the agent pool.
  • Stable, Rapid, or Target Kubernetes minor version channels with node image set to:
    • Consistent image: The node image upgrades to the version determined by Fleet Manager. The reference to the snapshot (creationData) is removed from the agent pool.
    • Latest image: The agent pool keeps its reference to the snapshot (creationData), and the node image isn't modified.

For more information, see understanding node image upgrades and snapshots.

Create auto-upgrade profiles

Create auto-upgrade profiles by using the az fleet autoupgradeprofile create command.

You can disable an auto-upgrade profile by including the --disabled flag with the az fleet autoupgradeprofile create command. To reenable a disabled auto-upgrade profile, rerun the az fleet autoupgradeprofile create command without the --disabled flag.

Note

Disabling an auto-upgrade profile doesn't affect any in-progress update runs. However, the process stops generating new update runs until you reenable the profile.

Stable channel Kubernetes updates

Update to the latest supported Kubernetes patch release on minor version N-1, where N is the latest supported minor version.

Update member clusters one by one

Update member clusters sequentially by using the az fleet autoupgradeprofile create command.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --channel Stable

Update member clusters by using an existing update strategy

Update member clusters by using an existing update strategy. Use the az fleet autoupgradeprofile create command with the --update-strategy-id parameter set to the ID of the existing update strategy.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --update-strategy-id $STRATEGYID \
  --channel Stable

Update member clusters by using an existing update strategy with consistent node image

Update member clusters by using an existing update strategy, and ensure the same node image version is used in every Azure region. Use the az fleet autoupgradeprofile create command with the --node-image-selection parameter set to Consistent. All member clusters run the same node image version.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --update-strategy-id $STRATEGYID \
  --channel Stable \
  --node-image-selection Consistent

Update member clusters by using an existing update strategy with latest node image

Update member clusters by using an existing update strategy, and ensure the latest available node image version is used for each Azure region. Use the az fleet autoupgradeprofile create command with the --node-image-selection parameter set to Latest. Member clusters can run multiple node image versions.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --update-strategy-id $STRATEGYID \
  --channel Stable \
  --node-image-selection Latest

Node image channel updates

Update nodes with a newly patched machine image that includes security fixes and bug fixes.

Update node images one by one

Update node images for member clusters sequentially by using the az fleet autoupgradeprofile create command.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --channel NodeImage

Update node images by using an existing update strategy

Update node images for member clusters by using an existing update strategy. Use the az fleet autoupgradeprofile create command and set the --update-strategy-id parameter to the ID of the existing update strategy.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --update-strategy-id $STRATEGYID \
  --channel NodeImage

Target Kubernetes minor version updates

Update to a defined target Kubernetes minor version by using the --target-kubernetes-version parameter. Supply the version in the {major version}.{minor version} format (for example, 1.33). Fleet Manager Auto-upgrade automatically upgrades member clusters to the latest patch release of the specified target version when the patch is available.

Important

Keep the following information in mind when using the Target Kubernetes minor version channel:

  • You must specify the --target-kubernetes-version parameter. This parameter isn't supported for other auto-upgrade channels (Rapid, Stable, Node image, and Security patch).
  • The long-term support (LTS) flag, --long-term-support, is only available when using the Target Kubernetes minor version channel. For other channels, set this flag to False.
  • You can only select LTS Kubernetes versions (N-2) for an auto-upgrade profile with the --long-term-support flag. For Fleet auto-upgrade to keep working in this scenario, ensure that the clusters in the generated update run are all enabled with LTS. Non-LTS clusters cause the update run to fail when the first non-LTS cluster is encountered.
  • You can't set the target Kubernetes version to a future Kubernetes version that AKS hasn't released.

Update member clusters to a specific Kubernetes minor version

Update member clusters to a specific Kubernetes minor version by using the az fleet autoupgradeprofile create command with the --target-kubernetes-version parameter set to the desired version. The following example updates member clusters to Kubernetes version 1.33.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --channel TargetKubernetesVersion \
  --target-kubernetes-version "1.33"

Update member clusters with LTS enabled to a specific Kubernetes minor version

Update member clusters with LTS enabled to a specific Kubernetes minor version by using the az fleet autoupgradeprofile create command with the --target-kubernetes-version parameter set to the desired version and the --long-term-support flag enabled. The following example updates member clusters to Kubernetes version 1.29 with LTS enabled.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --channel TargetKubernetesVersion \
  --target-kubernetes-version "1.29" \
  --long-term-support

Security patch channel updates

Update Linux nodes with security fixes. Apply updates by using live patching when possible. Otherwise, deploy a newly patched machine image that contains security fixes.

Update nodes with security patches one by one

Update node images for member clusters sequentially by using the az fleet autoupgradeprofile create command.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --channel SecurityPatch

Update nodes with security patches by using an existing update strategy

Update node images for member clusters by using an existing update strategy. Use the az fleet autoupgradeprofile create command and set the --update-strategy-id parameter to the ID of the existing update strategy.

az fleet autoupgradeprofile create \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --update-strategy-id $STRATEGYID \
  --channel SecurityPatch

View auto-upgrade profiles

Auto-upgrade profiles are Fleet-level resources. They aren't attached directly to individual AKS clusters. To determine which profiles can update your AKS clusters, first list the Fleet members and confirm that the clusters are members of the Fleet.

az fleet member list \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --query "[].{memberName:name,clusterResourceId:clusterResourceId}" \
  --output table

List all auto-upgrade profiles for the Fleet using the az fleet autoupgradeprofile list command.

az fleet autoupgradeprofile list \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --output table

Show a specific auto-upgrade profile for the Fleet using the az fleet autoupgradeprofile show command. Review the channel, status, and updateStrategyId in the output.

az fleet autoupgradeprofile show \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE

If updateStrategyId is empty, the profile uses the One by one update sequence and includes the Fleet's member clusters sequentially. If updateStrategyId contains a resource ID, show the referenced strategy to review the stages, groups, or member selectors that determine which Fleet members the generated update runs include.

PROFILE_STRATEGY_ID=$(az fleet autoupgradeprofile show \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE \
  --query updateStrategyId \
  --output tsv)

if [[ -n "$PROFILE_STRATEGY_ID" ]]; then
  az fleet updatestrategy show \
    --resource-group $GROUP \
    --fleet-name $FLEET \
    --name "${PROFILE_STRATEGY_ID##*/}"
fi

Delete an auto-upgrade profile

Use the az fleet autoupgradeprofile delete command to delete an existing auto-upgrade profile. After running this command, you're prompted to confirm deletion. To bypass the confirmation and immediately delete the profile, include --yes in the command.

az fleet autoupgradeprofile delete \
  --resource-group $GROUP \
  --fleet-name $FLEET \
  --name $AUTOUPGRADEPROFILE

Note

Deleting an auto-upgrade profile doesn't affect any in-progress update runs.

Create auto-upgrade profiles

The Terraform sample in this section creates a resource group, an Azure Kubernetes Fleet Manager (without a hub cluster), and a Fleet auto-upgrade profile. By default, the profile uses the Stable channel with Latest node image selection and is enabled. Set the auto_upgrade_channel, node_image_selection, auto_upgrade_disabled, target_kubernetes_version, and long_term_support variables to match the channel and options described in the previous sections. The sample doesn't set an update strategy, so the resulting profile always uses the One by one update sequence.

Review the Terraform code

Note

The sample code for this article is located in the Azure Terraform GitHub repo. View the log file containing the test results from current and previous versions of Terraform.

See more articles and sample code showing how to use Terraform to manage Azure resources.

  1. Create a directory to test the sample Terraform code, and make it the current directory.

  2. Create a file named providers.tf, and insert the following code:

    terraform {
  required_version = ">= 1.9.0"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
    azapi = {
      source  = "Azure/azapi"
      version = "~> 2.0"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.0"
    }
  }
}

provider "azurerm" {
  features {}
}

provider "azapi" {
}

  3. Create a file named variables.tf, and insert the following code:

    variable "location" {
  type        = string
  description = "Azure region where the resource group and Fleet Manager are created."
  default     = "eastus"
}

variable "resource_group_name" {
  type        = string
  description = "Name of the resource group. If null, a unique name is generated."
  default     = null
}

variable "fleet_name" {
  type        = string
  description = "Name of the Azure Kubernetes Fleet Manager. If null, a unique name is generated."
  default     = null
}

variable "auto_upgrade_profile_name" {
  type        = string
  description = "Name of the Fleet auto-upgrade profile. If null, a unique name is generated."
  default     = null
}

variable "auto_upgrade_channel" {
  type        = string
  description = "Auto-upgrade channel used by the profile."
  default     = "Stable"

  validation {
    condition     = contains(["Rapid", "Stable", "NodeImage", "TargetKubernetesVersion"], var.auto_upgrade_channel)
    error_message = "Auto-upgrade channel must be one of: Rapid, Stable, NodeImage, TargetKubernetesVersion."
  }
}

variable "target_kubernetes_version" {
  type        = string
  description = "Target Kubernetes version in major.minor format, for example 1.31. Required when auto_upgrade_channel is TargetKubernetesVersion; otherwise leave null."
  default     = null

  validation {
    condition     = var.target_kubernetes_version == null || can(regex("^[0-9]+\\.[0-9]+$", var.target_kubernetes_version))
    error_message = "Target Kubernetes version must be in major.minor format, for example 1.31."
  }

  validation {
    condition     = var.auto_upgrade_channel != "TargetKubernetesVersion" || var.target_kubernetes_version != null
    error_message = "Target Kubernetes version is required when auto_upgrade_channel is TargetKubernetesVersion."
  }
}

variable "node_image_selection" {
  type        = string
  description = "Node image selection type. Latest uses the newest image per region; Consistent waits for the same image version across all member regions."
  default     = "Latest"

  validation {
    condition     = contains(["Latest", "Consistent"], var.node_image_selection)
    error_message = "Node image selection must be either Latest or Consistent."
  }
}

variable "auto_upgrade_disabled" {
  type        = bool
  description = "Set to true to stop the profile from creating new automatic update runs. In-progress update runs aren't cancelled."
  default     = false
}

variable "long_term_support" {
  type        = bool
  description = "Whether the profile targets long-term support Kubernetes versions. Only supported on the TargetKubernetesVersion channel."
  default     = false

  validation {
    condition     = !var.long_term_support || var.auto_upgrade_channel == "TargetKubernetesVersion"
    error_message = "Long-term support can only be enabled when auto_upgrade_channel is TargetKubernetesVersion."
  }
}

variable "tags" {
  type        = map(string)
  description = "Tags to apply to created resources."
  default = {
    environment = "demo"
    managed_by  = "terraform"
  }
}

  4. Create a file named main.tf, and insert the following code:

    resource "random_string" "suffix" {
  length  = 5
  lower   = true
  numeric = true
  special = false
  upper   = false
}

locals {
  resource_group_name       = coalesce(var.resource_group_name, "rg-fleet-update-${random_string.suffix.result}")
  fleet_name                = coalesce(var.fleet_name, "fleet-update-${random_string.suffix.result}")
  auto_upgrade_profile_name = coalesce(var.auto_upgrade_profile_name, "stable-profile-${random_string.suffix.result}")

  # targetKubernetesVersion is only valid on the TargetKubernetesVersion channel.
  auto_upgrade_properties = merge(
    {
      channel         = var.auto_upgrade_channel
      disabled        = var.auto_upgrade_disabled
      longTermSupport = var.long_term_support
      nodeImageSelection = {
        type = var.node_image_selection
      }
    },
    var.auto_upgrade_channel == "TargetKubernetesVersion" ? {
      targetKubernetesVersion = var.target_kubernetes_version
    } : {}
  )
}

resource "azurerm_resource_group" "example" {
  name     = local.resource_group_name
  location = var.location
  tags     = var.tags
}

resource "azurerm_kubernetes_fleet_manager" "example" {
  name                = local.fleet_name
  location            = azurerm_resource_group.example.location
  resource_group_name = azurerm_resource_group.example.name
  tags                = var.tags
}

# autoUpgradeProfiles is only available on the preview API version, which requires schema validation to be disabled.
resource "azapi_resource" "auto_upgrade_profile" {
  type                      = "Microsoft.ContainerService/fleets/autoUpgradeProfiles@2025-04-01-preview"
  name                      = local.auto_upgrade_profile_name
  parent_id                 = azurerm_kubernetes_fleet_manager.example.id
  schema_validation_enabled = false

  body = {
    properties = local.auto_upgrade_properties
  }
}

  5. Create a file named outputs.tf, and insert the following code:

    output "resource_group_name" {
  description = "Name of the created resource group."
  value       = azurerm_resource_group.example.name
}

output "fleet_manager_name" {
  description = "Name of the Azure Kubernetes Fleet Manager."
  value       = azurerm_kubernetes_fleet_manager.example.name
}

output "fleet_manager_id" {
  description = "Resource ID of the Azure Kubernetes Fleet Manager."
  value       = azurerm_kubernetes_fleet_manager.example.id
}

output "auto_upgrade_profile_name" {
  description = "Name of the Fleet auto-upgrade profile."
  value       = azapi_resource.auto_upgrade_profile.name
}

output "auto_upgrade_profile_id" {
  description = "Resource ID of the Fleet auto-upgrade profile."
  value       = azapi_resource.auto_upgrade_profile.id
}

  6. Copy the sample's terraform.tfvars.example file to terraform.tfvars in the same directory, and then edit the values to match the auto-upgrade channel, node image selection, and other options you want.

    cp terraform.tfvars.example terraform.tfvars

Initialize, validate, and apply the configuration

Run terraform init to initialize the Terraform deployment. This command downloads the Azure providers required to manage your Azure resources.

terraform init

Run terraform fmt to format the configuration files consistently.

terraform fmt

Run terraform validate to confirm that the configuration files are syntactically valid.

terraform validate

Run terraform plan to create an execution plan.

terraform plan -out main.tfplan

Run terraform apply to apply the execution plan to your cloud infrastructure.

terraform apply main.tfplan

View auto-upgrade profiles

Get the resource group, Fleet Manager, and auto-upgrade profile names from the Terraform outputs.

resource_group_name=$(terraform output -raw resource_group_name)
fleet_manager_name=$(terraform output -raw fleet_manager_name)
auto_upgrade_profile_name=$(terraform output -raw auto_upgrade_profile_name)

Show the auto-upgrade profile's channel, status, and update sequence by using the az fleet autoupgradeprofile show command.

az fleet autoupgradeprofile show \
  --resource-group $resource_group_name \
  --fleet-name $fleet_manager_name \
  --name $auto_upgrade_profile_name

The sample doesn't set an update strategy, so updateStrategyId in the output is empty and the profile updates the Fleet's member clusters sequentially using the One by one update sequence. List the current Fleet members with the following command.

az fleet member list \
  --resource-group $resource_group_name \
  --fleet-name $fleet_manager_name \
  --output table

Delete an auto-upgrade profile

Run terraform plan with the -destroy flag to create a destroy execution plan.

terraform plan -destroy -out main.destroy.tfplan

Run terraform apply to apply the destroy plan.

terraform apply main.destroy.tfplan

Warning

This command deletes the auto-upgrade profile, the Azure Kubernetes Fleet Manager, and the resource group created by this sample, not just the profile. If you joined AKS clusters as members of the Fleet, remove them as Fleet members first. Confirm the resource group doesn't contain other resources you want to keep before you run this command.

Create auto-upgrade profiles

Update to the latest supported Kubernetes patch release on minor version N-1, where N is the latest supported minor version.

  1. In the Azure portal, navigate to your Azure Kubernetes Fleet Manager resource.

  2. From the service menu, under Settings, select Multicluster update > Auto-upgrade profiles > + Create.

  3. On Create an auto-upgrade profile, configure the following options:

    • Under Auto-upgrade details:
      • Auto-upgrade profile name: Enter a name for the auto-upgrade profile.
      • Status: Select Enabled or Disabled. Disabled auto-upgrade profiles don't trigger when new versions are released.
      • Update sequence: Select Stages or One by one.
    • Under Auto-upgrade triggers:
      • Channel: Select Stable.
      • Node image: Select Latest image or Consistent image.
    • Under Strategy details:
      • If you selected an Update sequence using Stages, select an existing strategy or create a new one.

  4. Select Create to create the auto-upgrade profile.

    Screenshot of the Azure Kubernetes Fleet Manager Azure portal pane for creating auto-upgrade profile using the Stable channel.

View auto-upgrade profiles

Auto-upgrade profiles are Fleet-level resources. They aren't attached directly to individual AKS clusters. To determine which profiles can update your AKS clusters, confirm that the clusters are members of the Fleet, and then review the Fleet's profiles and their update sequences.

  1. In the Azure portal, navigate to your Azure Kubernetes Fleet Manager resource.
  2. From the service menu, select Fleet members, and confirm that your AKS clusters appear in the member list.
  3. Under Settings, select Multicluster update > Auto-upgrade profiles.
  4. Select an auto-upgrade profile to view its channel, status, and update sequence.
  5. If the update sequence is One by one, the profile includes the Fleet's member clusters sequentially. If the update sequence uses Stages, review the associated strategy's stages, groups, and member selectors to determine which Fleet members the generated update runs include.

Delete an auto-upgrade profile

  1. In the Azure portal, navigate to your Azure Kubernetes Fleet Manager resource.
  2. From the service menu, under Settings, select Multicluster update > Auto-upgrade profiles.
  3. Select the profile you want to delete, and then select Delete > Yes to confirm.

Note

Deleting an auto-upgrade profile doesn't affect any in-progress update runs.

Validate auto-upgrades

Auto-upgrades happen only when new Kubernetes or node images are available. When auto-upgrade is triggered, a linked update run is created, so you can use manage update run to see the results of the auto-upgrade.

You can also check your existing versions as a baseline.

Get the current Kubernetes version for a member cluster by using the az aks show command with the --query parameter to filter the output for the currentKubernetesVersion property.

az aks show \
  --resource-group $GROUP \
  --name $CLUSTER \
  --query currentKubernetesVersion

Get the current node image version for a member cluster by using the az aks show command with the --query parameter to filter the output for the nodeImageVersion property.

az aks show \
  --resource-group $GROUP \
  --name $CLUSTER \
  --query "agentPoolProfiles[].{name:name,mode:mode, nodeImageVersion:nodeImageVersion, osSku:osSku, osType:osType}"

When update runs finish, you can rerun these commands and view the updated versions that are deployed.

Generate an update run from an auto-upgrade profile

After you create an auto-upgrade profile, some time might pass before a new Kubernetes or node image version triggers auto-upgrade to create and execute an update run. Auto-upgrade allows you to generate a new update run at any time by using the az fleet autoupgradeprofile generate-update-run command. The resulting update run is based on the current AKS-published Kubernetes or node image version.

For more information about creating an on-demand update run from an auto-upgrade profile, see Generate an update run from an auto-upgrade profile.

Related content

  • Last updated on