Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Intel Advanced Matrix Extensions (Intel AMX) is a built-in AI accelerator in Intel Xeon Scalable processors. It's an extension to the x86 instruction set that accelerates the matrix-multiplication operations at the heart of deep learning, so you can run AI inference and training directly on the CPU without a discrete accelerator such as a GPU. Intel introduced Intel AMX with 4th Generation Intel Xeon Scalable processors (Sapphire Rapids).
On Azure, Intel AMX is available on virtual machines (VMs) that use these newer Intel processors. Because it's a CPU instruction-set feature rather than a separate device, there's no driver to install. You benefit from Intel AMX by running an AI framework or library that targets it, and it works on both Linux and Windows.
Key capabilities of Intel AMX
Intel AMX adds dedicated matrix-processing hardware to each CPU core. Its main elements include:
- Tiles: A set of two-dimensional registers that hold large blocks of matrix data close to the core.
- Tile Matrix Multiplication (TMUL): An accelerator engine that operates on the tiles to perform matrix-multiply operations in a single instruction.
- BF16 and INT8 data types: Support for bfloat16 (training and inference) and 8-bit integer (inference) precision, which speeds up AI math while reducing memory use.
- Framework integration: Optimizations upstreamed into popular frameworks and libraries such as PyTorch, TensorFlow, and Intel oneDNN, so you can use Intel AMX with minimal code changes.
Azure VM sizes with Intel AMX support
Intel AMX is available on every Azure VM size that uses 4th Generation Intel Xeon Scalable processors (Sapphire Rapids) or a newer Intel generation, such as 5th Generation (Emerald Rapids) and 6th Generation (Granite Rapids). That covers a large and growing number of series, including most current-generation Intel general-purpose and memory-optimized sizes such as the Dsv6, Esv6, Dsv7, and Esv7 series.
Because so many sizes qualify, here's a list of sizes that don't have Intel AMX:
| VM sizes | Why Intel AMX isn't available |
|---|---|
| Intel sizes on older processor generations, such as the Dsv5 and Esv5 series (Intel Ice Lake) and v4 and earlier series (Intel Cascade Lake) | These processors predate Intel AMX, which was introduced with 4th Generation Intel Xeon Scalable (Sapphire Rapids). |
| AMD-based sizes, such as the Dasv6 and Easv6 series (AMD EPYC) | Intel AMX is an Intel-only instruction-set extension. |
| Arm-based sizes, such as the Azure Cobalt 100 (Dpsv6, Epsv6) and Ampere Altra (Dpsv5, Epsv5) series | Intel AMX is an Intel x86-only feature. |
To confirm whether a specific size includes Intel AMX, check the processor listed on its size page. If the size uses a 4th Generation Intel Xeon (Sapphire Rapids) processor or newer, it supports Intel AMX. For the full list of VM sizes, see the VM sizes overview.
Verify Intel AMX is available
After you deploy a VM, confirm that Intel AMX is present on the CPU. On Linux, check the CPU flags:
lscpu | grep amx
If Intel AMX is available, the output lists the amx_tile, amx_bf16, and amx_int8 flags.
Use Intel AMX
Intel AMX is a CPU feature, so there's no driver to install. To take advantage of it, run AI frameworks and libraries that are optimized for Intel AMX. Intel upstreams these optimizations into popular open-source projects, so in many cases you only need a recent framework version.
For Intel's own documentation and getting-started resources, see: