Using Flex ScaleSets
As of 8.3.0, CycleCloud can use Flex orchestration for scale sets. This works differently than the automatic usage of Uniform scale sets that is standard in CycleCloud. In this mode, you create a Flex scale set outside of CycleCloud, and you specify which nodes should use it. CycleCloud creates and deletes VMs in that scale set. This works for both head nodes and execute nodearrays.
To use Flex orchestration, you must use a CycleCloud credential that is locked to a given resource group (which must be created). This is because VMs in a Flex scale set must be in the same resource group as the scale set. You can use the az CLI to create the resource group, if you don't have one to use already:
az group create --location REGIONNAME --resource-group RESOURCEGROUP
The scaleset must be created in Flex orchestration mode, and any VM settings on it (e.g., VM size or image) are ignored. Because of this, it is easiest to create it through the az CLI:
az vmss create --orchestration-mode Flexible --resource-group RESOURCEGROUP --name SCALESET --platform-fault-domain-count 1
Finally, specify the fully qualified id for this scaleset on the node or nodearray that should use it on the cluster template:
[nodearray execute] FlexScaleSetId = /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/RESOURCEGROUP/providers/Microsoft.Compute/virtualMachineScaleSets/SCALESET
Scale sets have limitations on size (currently 1000 VMs). To scale larger than that, you must create multiple scale sets and assign them to different nodearrays.