Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Before you create a cluster in Azure CycleCloud, plan how the cluster is structured and sized. Good planning decisions up front - about the scheduler, the virtual machine (VM) types, how the cluster scales, and how it stores and moves data - determine how well the cluster performs and how much it costs to run.
This article describes the main decisions to make when you plan an HPC cluster in CycleCloud. It complements CycleCloud clusters and nodes, which explains what clusters, nodes, and node arrays are, and Create a new cluster, which walks through building a cluster from a template.
Confirm that CycleCloud fits your workload
Use CycleCloud when you need to operate a specific HPC scheduler, customize the cluster topology and software stack, or align closely with existing on-premises workflows. Before you size a custom cluster, consider these alternatives:
- Choose Azure CycleCloud Workspace for Slurm when you want a ready-to-deploy Slurm environment with networking, storage, and access components provisioned in your subscription.
- Choose Azure Batch when you want Azure to provide scheduling as a service and your application can use Batch pools, jobs, and tasks.
If CycleCloud is the right operating model, use the following sections to create a planning baseline. Record your scheduler, VM, scale, quota, region, storage, network, and cost decisions before deployment, and validate them with a representative workload.
Choose a scheduler
CycleCloud is scheduler-agnostic. It has built-in support for the schedulers that most HPC teams already use, and it adds autoscaling to each of them:
- Slurm
- PBS Professional
- IBM Spectrum LSF
- Altair Grid Engine
- HTCondor
Choose the scheduler that matches your existing workflows and job scripts so that you can move workloads to Azure with minimal change. If your organization uses a scheduler that isn't in the built-in list, you can integrate it with the CycleCloud autoscaling API. For more information, see Scheduling in CycleCloud.
Choose VM families and sizes
The VM family and size you select for compute nodes have the largest effect on both performance and cost. Match the VM to the characteristics of your workload:
- Tightly coupled (MPI) workloads that exchange data between nodes benefit from HPC-optimized VMs with a low-latency InfiniBand interconnect, such as the HPC VM sizes.
- GPU-accelerated workloads, such as AI training or rendering, use GPU-optimized VM sizes.
- Embarrassingly parallel or throughput workloads, where tasks run independently, can use general-purpose or compute-optimized VMs and don't require a specialized interconnect.
For a full comparison of VM options, see Sizes for virtual machines in Azure.
Because a node array can span more than one virtual machine scale set, a single array can offer VMs of different sizes or families for the same role. This flexibility helps when your preferred VM size has limited capacity in a region. For more information, see CycleCloud clusters and nodes.
Size the cluster and plan for autoscaling
CycleCloud automatically scales the number of compute nodes based on the work queued in the scheduler, so you don't need to guess a fixed cluster size. Instead, plan the following boundaries:
- Maximum size. The cluster creation form limits how far a cluster autoscales, based on the total number of nodes or cores it can start. Set this limit to reflect the largest workload you expect and the quota available in the region.
- Dedicated versus Spot nodes. Dedicated nodes are reserved for your pool and give you predictable capacity. Spot nodes use surplus Azure capacity at a reduced price, but Azure can reclaim them when it needs the capacity back. Use dedicated nodes for time-sensitive or long-running jobs, and Spot nodes for fault-tolerant or restartable work where cost matters more than guaranteed availability.
- Scale-down behavior. Autoscaling removes idle nodes so that you stop paying for compute you aren't using. Confirm that your jobs write results to durable storage, because local data on a node is lost when the node is deallocated.
Understand autoscaling limits
For a CycleCloud cluster, several independent limits determine how many nodes CycleCloud can start. The lowest effective limit wins:
| Limit | What it controls |
|---|---|
| Available total regional vCPU quota | The remaining vCPUs that your subscription can deploy across all VM families in the region. |
| Available VM-family vCPU quota | The remaining vCPUs that your subscription can deploy for the selected VM family in the region. Both vCPU quotas must allow the deployment. |
CycleCloud MaxCount |
The maximum number of nodes in a node array. |
CycleCloud MaxCoreCount |
The maximum number of cores in a node array. If a template defines both MaxCount and MaxCoreCount, CycleCloud applies the lower effective constraint. |
Important
Quota and CycleCloud limits are ceilings, not capacity reservations. Actual provisioning can be lower because of temporary Azure capacity constraints or configuration errors.
To identify a constraint, go to the CycleCloud cluster page for your cluster. In the Nodes table, select Actions > Add. The Add Nodes dialog describes the applicable constraints for the selected node array.
For example, suppose a node array uses current-generation Standard_HB368rs_v5 VMs, which consume 368 vCPUs each, and has these limits:
| Constraint | Available or configured limit | Effective nodes |
|---|---|---|
| Total regional vCPU quota | 3,680 vCPUs | 10 |
| HBv5 VM-family vCPU quota | 2,208 vCPUs | 6 |
CycleCloud MaxCount |
8 nodes | 8 |
The HBv5 VM-family quota is the lowest limit, so the configured ceiling is six nodes even though MaxCount allows eight. If Azure has capacity for only five VMs when the request runs, CycleCloud provisions five nodes and the remaining request stays constrained until capacity becomes available. Increasing MaxCount alone doesn't raise either the quota ceiling or available capacity.
When a template uses MaxCoreCount, divide that value by the VM's vCPU count and round down to find its node ceiling. For definitions of both CycleCloud settings, see Node and node array objects.
Check quota, capacity, and region
The VMs you plan to use must be available, and within quota, in the region where you deploy:
- Region. Choose a region that offers the VM families your workload needs and that's close to your data and users. Not every VM family is available in every region.
- Quota. Core (vCPU) quotas are set per region and per VM family. Make sure your quota is high enough for the maximum cluster size you planned, and request a quota increase if needed.
- Capacity. Even within quota, HPC VM sizes can be constrained in a region at a given time. Consider a fallback VM size or an additional region for large or time-critical runs.
Plan storage and networking
HPC jobs are often bound by how quickly they can read and write data, not just by compute:
- Shared file systems. Most HPC clusters need a shared file system that all nodes can mount. CycleCloud can deploy NFS servers and parallel file systems (for example, BeeGFS) as part of the cluster, and you can connect to managed services such as Azure NetApp Files or Azure Managed Lustre.
- Interconnect. Tightly coupled MPI jobs require the InfiniBand interconnect available on HPC-optimized VMs. Confirm that both the VM size and the region support it.
- Networking. Plan the virtual network and subnets that host the cluster, and ensure the CycleCloud application server has the outbound access it needs to manage nodes. For production guidance, see Plan your production deployment.
Choose a cluster data path
How you get data onto compute nodes has a large effect on throughput and cost. Choose based on how your jobs read and write data:
| Option | Best for | Notes |
|---|---|---|
| BlobFuse2 (streaming) | Reading large datasets straight from Blob (AI training data, simulation inputs) at high throughput, with no standing file server | Per-node FUSE mount; not fully POSIX. See Mount Azure Blob Storage on cluster nodes. |
| Azure Managed Lustre | Tightly coupled or I/O-intensive jobs that need a shared, POSIX parallel file system; can hydrate from Blob | Managed parallel file system. See Azure Managed Lustre. |
| Azure HPC Cache | Caching an existing NAS or Blob back end for read-heavy HPC, including hybrid and on-premises data | Distributed cache tier in front of storage. See Azure HPC Cache. |
| Azure NetApp Files | Low-latency shared NFS or SMB for EDA and general HPC home and scratch space | Managed file service. See Azure NetApp Files. |
As a rule of thumb: use BlobFuse2 streaming when your data already lives in Blob and jobs mostly read it. Use a shared or parallel file system (Managed Lustre or NetApp Files) when jobs need cross-node coordination, POSIX semantics, or heavy shared writes. Use a cache tier (HPC Cache) when you need to accelerate access to an existing back end.
Plan for cost
Compute costs are usually the largest expense in an HPC environment. Plan to control these costs by:
- Using autoscaling and Spot nodes to avoid paying for idle capacity.
- Right-sizing VMs to match the workload instead of defaulting to the largest option.
- Tracking spending through the built-in integration with Microsoft Cost Management. For more information, see Cost and usage tracking.
Validate the cluster before production
Run a proof of concept with a representative job, its expected data volume, and the production scheduler and cluster configuration. Don't approve the design for production until you can record evidence for every criterion:
| Criterion | Evidence to record |
|---|---|
| Workflow compatibility | Existing job scripts, scheduler policies, software, licenses, and identity integrations work without unplanned changes. |
| Scale and quota | Autoscaling reaches the required node count in the intended region without exceeding total regional, VM-family, MaxCount, or MaxCoreCount limits. |
| Compute and storage performance | End-to-end job time meets the target, including queue time, node provisioning, data staging, compute, and result persistence. |
| Recovery | After a node provisioning failure, job failure, or Spot eviction if used, the workload stays within the recorded maximum recovery time and permitted checkpoint loss, meets the completion deadline, and successfully uses the planned fallback VM size. |
| Cost | Measured compute, storage, networking, and license cost per completed job meets the target at the expected run frequency. |
| Operations | Named owners can monitor the scheduler and infrastructure, investigate failed scale requests, patch the environment, and recover jobs and data. |
If a criterion fails, revise the VM, node-array limits, storage path, autoscaling configuration, or regional fallback and repeat the same job. A small functional test doesn't establish production readiness because it doesn't exercise scheduler compatibility, peak scale, or data movement.