Azure ML compute cluster nodes provision with outdated Ubuntu node image (kernel 6.8.0-1059-azure) - how to force a newer node image?

Question

I'm running Azure Machine Learning compute clusters (AmlCompute) in West Europe. The nodes provision with an Ubuntu 22.04 host image whose kernel is 6.8.0-1059-azure, built 2026-05-28.

A security scan flagged packages on this node image that are fixed in newer Ubuntu updates. Ubuntu has already published newer versions (e.g. linux-azure kernel 6.8.0-1063.71 in updates/security on 2026-07-23), but the AML nodes still come up on the older 6.8.0-1059-azure image.

I tried the usual remediation - deleting and recreating the cluster - but a brand-new AmlCompute cluster still allocates nodes on the exact same 6.8.0-1059-azure (2026-05-28) image. So recreating the cluster does not pick up a newer OS.

Questions:
1. How is the host OS/node image for AmlCompute nodes updated, and on what cadence does the platform refresh it (West Europe)?
2. Is there any supported way to force AML compute nodes to use a newer node image (a setting, or a specific redeploy method), or do we have to wait for the platform to republish the node image?
3. Is there a way to check which node-image version a given region/pool currently serves?

Environment:
- Azure Machine Learning, AmlCompute cluster
- Region: West Europe
- Node OS: Ubuntu 22.04.5 LTS, kernel 6.8.0-1059-azure (#65~22.04.1-Ubuntu SMP Thu May 28 16:59:19 UTC 2026)

Thanks!

Answer 1

You’re encountering the managed-image boundary, not a cluster setting problem. Azure Machine Learning controls the AmlCompute host OS image, updates it monthly from Canonical packages, and can roll out compute-cluster hotfixes within 72 hours for qualifying vulnerabilities. A newer Ubuntu package upstream does not mean AML’s validated image has reached West Europe.

Set minimum nodes to zero, allow jobs to finish, and wait until the cluster fully scales to zero. The next allocation uses the latest AML image available in that pool. Deleting and recreating cannot select an image the service has not published, and AmlCompute has no supported host-image or custom-image override.

After reallocation, verify uname -r and package versions. Microsoft does not publish a per-region AmlCompute image catalog. Since a new cluster still receives 6.8.0-1059, open Microsoft Support with the CVEs, cluster ID, region, and provisioning timestamps so the service team can confirm rollout status or hotfix eligibility.

Answer 2

Hello @Itamar Latnik ,

Welcome to Microsoft Q&A .Thank you for reaching out to us.

After reviewing the scenario, the behavior is consistent with the way Azure Machine Learning manages AmlCompute host images. Azure ML compute nodes are provisioned from validated, platform-managed images rather than directly from the latest Ubuntu repository packages. Therefore, newer Ubuntu security updates may become available before they are integrated, validated and rolled out as part of an Azure ML managed image for a specific region.

Regarding AmlCompute host images updation and refresh cadence -

The host operating system used by AmlCompute is fully managed by Azure Machine Learning. The underlying node image is not customer-managed and cannot be directly patched or updated.

The image lifecycle works as follows:

• Azure Machine Learning periodically rebuilds and validates managed VM images using updates from Canonical (Ubuntu) along with additional platform servicing and validation.
• Managed images follow a regular target servicing cadence, typically monthly, with expedited servicing for certain security issues where applicable.
• Updated images are rolled out progressively across Azure regions after validation.
• Ubuntu repositories and Azure Machine Learning managed images follow separate release processes. As a result, a newer Ubuntu kernel or package may be available before it becomes part of an Azure ML managed image.
• There is no published SLA or fixed regional release schedule indicating exactly when a particular Azure region will receive a refreshed AmlCompute image.

For this reason, a difference between the latest Ubuntu package version and the kernel version present on a newly provisioned AmlCompute node can occur until the updated managed image completes validation and regional rollout.

Regarding if there is a supported way to force AmlCompute to use a newer node image -

Currently, there is no supported mechanism to force AmlCompute to provision a specific or newer host image.

There is no supported option to select:

• Ubuntu version
• Kernel version
• Host image version
• Image SKU
• Specific AmlCompute image revision

Operations such as:

• deleting and recreating a compute cluster,
• scaling the cluster to zero,
• provisioning additional nodes,

will provision nodes using the latest Azure Machine Learning managed image currently available in that region.Currently, there is no supported mechanism to force AmlCompute to provision a specific or newer host image.

There is no supported option to select:

• Ubuntu version
• Kernel version
• Host image version
• Image SKU
• Specific AmlCompute image revision

Operations such as:

• deleting and recreating a compute cluster,
• scaling the cluster to zero,
• provisioning additional nodes,

will provision nodes using the latest Azure Machine Learning managed image currently available in that region.

Scaling to zero is useful because it allows future node provisioning to consume an updated image when one is available. However, scaling to zero does not trigger publication of a new image or force an image refresh.

Since a new compute cluster has already been created and the same kernel version was observed, the current evidence indicates that the managed image available in the West Europe region has not yet changed.

Recommended validation steps:

Review compute cluster scaling configuration:

• min_instances
• idle_time_before_scale_down

If operationally possible:

• Allow active workloads to complete.
• Allow the cluster to fully deallocate.
• Provision fresh nodes.
• Validate the node image details:

uname -r
uname -v
cat /etc/os-release
cat /proc/version

Regarding if there is a way to determine which image version is currently served in a region

Currently, there is no supported CLI command, REST API, Azure portal view, or public regional catalog that exposes the exact AmlCompute host image revision currently served in a region.

The practical method is to inspect a newly provisioned node:

uname -r
uname -v
cat /etc/os-release
cat /proc/version

Recommended Additional Validation

Before determining the remediation path, it is important to confirm whether the reported vulnerabilities are related to the host operating system or the workload environment.

Azure Machine Learning workloads commonly execute inside Docker-based environments. Therefore:

• Host OS vulnerabilities require an updated Azure ML managed node image.
• Container vulnerabilities require updating the workload environment or container image.

Updating the host kernel will not resolve vulnerabilities that exist only inside a container image.

Recommended checks:

Confirm the source of the vulnerability findings:

• Host operating system
• Docker/container image

Validate the reported CVEs against installed package versions.

Collect the following information:

uname -a
uname -r
uname -v
cat /etc/os-release
cat /proc/version
dpkg -l | grep linux

The following references might be helpful , please check them out

• Create compute clusters - Azure Machine Learning | Microsoft Learn
• Vulnerability management - Azure Machine Learning | Microsoft Learn
• GitHub - Azure/AzureML-Containers: Docker containers for running training scripts on AzureML · GitHub
• Manage Azure Machine Learning environments with the CLI & SDK (v2) - Azure Machine Learning | Microsoft Learn
• About Azure Machine Learning environments - Azure Machine Learning | Microsoft Learn

Please let us know if the response was helpful

 

Thank you