Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Azure Managed Lustre is a managed parallel file system for high-performance computing (HPC) and artificial intelligence (AI) workloads. It provides shared file storage while Azure manages the underlying file system infrastructure.
On Azure Kubernetes Service (AKS), the Azure Managed Lustre Container Storage Interface (CSI) driver makes this storage available to Linux workloads through persistent volumes and persistent volume claims. Multiple pods can access a shared file system using ReadWriteMany volumes.
When to use Azure Managed Lustre
Consider Azure Managed Lustre when your containerized workloads need high aggregate file-system bandwidth across many workers:
- AI training data access: Read shared datasets concurrently to supply distributed training jobs.
- Checkpointing and recovery: Write model state and read checkpoints when restarting a job.
- HPC and data processing: Share input datasets and output files across parallel compute tasks.
You can also integrate the file system with Azure Blob Storage to import datasets and export results. For details, see Azure Blob Storage integration.
Performance and scale
Azure Managed Lustre supports petabyte-scale file systems, and published benchmark configurations demonstrate aggregate throughput of hundreds of gigabytes per second. Size the file system for both data capacity and workload bandwidth; the throughput tier and provisioned capacity determine its nominal bandwidth.
For example, the documented maximum for the 20 MBps tier without Blob integration is 25,632 TiB, approximately 25 PiB. Limits differ by tier and Blob integration configuration, and exceeding a subscription's default maximum requires a support request. Use the Azure Managed Lustre scale targets for the current limits and capacity increments rather than treating a benchmark configuration as a deployment limit.
MLPerf Storage results
The published Azure Managed Lustre MLPerf Storage v3.0 results include these examples:
| Workload | Tested configuration | Measured aggregate bandwidth |
|---|---|---|
| Training reads | AMLFS 125, 4,096 TiB, 70 simulated B200 accelerators | 379.1 GiB/s, or 407.1 GB/s. |
| Checkpoint writes and recovery reads | AMLFS 125, 4,096 TiB, 1250B-parameter checkpoint workload | 642.2 GiB/s writes and 489.6 GiB/s recovery reads, or 689.6 GB/s and 525.7 GB/s, respectively. |
MLPerf Storage exercises the storage data path with simulated accelerators. These figures are storage-system results, not per-pod throughput or measurements of AKS CSI-driver overhead, GPU computation, or end-to-end training time. GiB/s uses binary units; GB/s uses decimal units.
Benchmark results describe the tested configurations, not a performance guarantee. Validate your application with its intended data layout, concurrency, AKS node networking, and file system size. See the complete results and sizing guidance and performance optimization guidance.
Use Azure Managed Lustre with AKS
Install the CSI driver with Helm or a script. Your workloads use Kubernetes storage classes and claims to access the file system:
- Static provisioning connects an existing Azure Managed Lustre file system.
- Dynamic provisioning creates a file system for a persistent volume claim.
An alternative AKS CLI installation option (preview) installs the driver as a managed extension. Neither installation method changes the service's capacity or performance limits.