Using managed identities

Use Microsoft Entra managed identities to give Azure CycleCloud permission to manage clusters in your subscription. This approach serves as an alternative to using a service principal. Assign managed identities to CycleCloud VMs to provide access to Azure resources like Storage, Key Vault, or Azure Container Registries.

CycleCloud VM permissions with managed identity

CycleCloud automates many calls to the Azure Resource Manager to manage HPC clusters. This automation needs certain permissions for CycleCloud. You can give CycleCloud this access by setting up a Service Principal or by assigning a Managed Identity to the CycleCloud VM.

We recommend using either a System-Assigned or User-Assigned Managed Identity to grant these permissions instead of a Service Principal.

When you install Azure CycleCloud on an Azure VM and assign a Managed Identity to the VM, the Add Subscription dialog works a little differently. The dialog enables and preselects the Managed Identity authentication option. It also fills in the Subscription ID with the subscription for the host VM.

You can still enter the standard set of credentials by selecting the App Registration authentication option. When you select this option, the standard fields appear in the form. You can also use a different Subscription ID; the provided value is just for convenience.

When you use a System Assigned Managed Identity, leave the ClientID field blank. But when you use CycleCloud with a User-Assigned Managed Identity, set the ClientID to the ClientID of the specific Managed Identity you want for cluster orchestration.

Storage locker access

In addition to using a managed identity for cluster orchestration on the CycleCloud VM, you can configure CycleCloud to assign a user-assigned managed identity to clusters for storage account and locker access from cluster nodes. This approach uses the user-assigned managed identity instead of SAS tokens derived from the storage account's shared access key.

To configure clusters to use a user-assigned managed identity rather than the shared access key, create a dedicated user-assigned managed identity with Storage Blob Data Reader access at the storage account scope. First, create the storage account and user-assigned managed identity in your Azure subscription. Then, in the Storage Locker Configuration section of the Add Subscription dialog, select the new managed identity from the Locker Identity dropdown and the storage account from the Storage Account dropdown.

Create a custom role and managed identity for CycleCloud

The simplest option that provides sufficient access rights is to assign the Contributor and Storage Blob Data Contributor roles for the subscription to the CycleCloud VM as a system-assigned managed identity. However, the Contributor role has a higher privilege level than CycleCloud requires. You can create and assign a custom role to the VM. Similarly, assign the Storage Blob Data Contributor role at the storage account scope rather than subscription scope if you already created the storage account.

This role covers all CycleCloud features:

{
    "assignableScopes": [
      "/subscriptions/<SubscriptionId>"
    ],
    "description": "CycleCloud Orchestrator Role",
    "permissions": [
      {
        "actions": [
          "Microsoft.Authorization/*/read",
          "Microsoft.Authorization/roleAssignments/*",
          "Microsoft.Authorization/roleDefinitions/*",
          "Microsoft.Commerce/RateCard/read",
          "Microsoft.Compute/*/read",
          "Microsoft.Compute/availabilitySets/*",
          "Microsoft.Compute/disks/*",
          "Microsoft.Compute/images/read",
          "Microsoft.Compute/locations/usages/read",
          "Microsoft.Compute/register/action",
          "Microsoft.Compute/skus/read",
          "Microsoft.Compute/virtualMachines/*",
          "Microsoft.Compute/virtualMachineScaleSets/*",
          "Microsoft.Compute/virtualMachineScaleSets/virtualMachines/*",
          "Microsoft.ManagedIdentity/userAssignedIdentities/*/read",
          "Microsoft.ManagedIdentity/userAssignedIdentities/*/assign/action",
          "Microsoft.MarketplaceOrdering/offertypes/publishers/offers/plans/agreements/read",
          "Microsoft.MarketplaceOrdering/offertypes/publishers/offers/plans/agreements/write",
          "Microsoft.Network/*/read",
          "Microsoft.Network/locations/*/read",
          "Microsoft.Network/networkInterfaces/read",
          "Microsoft.Network/networkInterfaces/write",
          "Microsoft.Network/networkInterfaces/delete",
          "Microsoft.Network/networkInterfaces/join/action",
          "Microsoft.Network/networkSecurityGroups/read",
          "Microsoft.Network/networkSecurityGroups/write",
          "Microsoft.Network/networkSecurityGroups/delete",
          "Microsoft.Network/networkSecurityGroups/join/action",
          "Microsoft.Network/publicIPAddresses/read",
          "Microsoft.Network/publicIPAddresses/write",
          "Microsoft.Network/publicIPAddresses/delete",
          "Microsoft.Network/publicIPAddresses/join/action",
          "Microsoft.Network/register/action",
          "Microsoft.Network/virtualNetworks/read",
          "Microsoft.Network/virtualNetworks/subnets/read",
          "Microsoft.Network/virtualNetworks/subnets/join/action",
          "Microsoft.Resources/deployments/read",
          "Microsoft.Resources/subscriptions/resourceGroups/read",
          "Microsoft.Resources/subscriptions/resourceGroups/write",
          "Microsoft.Resources/subscriptions/resourceGroups/delete",
          "Microsoft.Resources/subscriptions/resourceGroups/resources/read",
          "Microsoft.Resources/subscriptions/operationresults/read",
          "Microsoft.Storage/*/read",
          "Microsoft.Storage/checknameavailability/read",
          "Microsoft.Storage/register/action",
          "Microsoft.Storage/storageAccounts/blobServices/containers/delete",
          "Microsoft.Storage/storageAccounts/blobServices/containers/read",
          "Microsoft.Storage/storageAccounts/blobServices/containers/write",
          "Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action",
          "Microsoft.Storage/storageAccounts/read",
          "Microsoft.Storage/storageAccounts/listKeys/action",
          "Microsoft.Storage/storageAccounts/write"
        ],
        "dataActions": [
          "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete",
          "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read",
          "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write",
          "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/move/action",
          "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/add/action"
        ],
        "notActions": [],
        "notDataActions": []
      }
    ],
    "Name": "CycleCloud <SubscriptionId>",
    "roleType": "CustomRole",
    "type": "Microsoft.Authorization/roleDefinitions"
}

Make sure to replace <SubscriptionId> with your subscription ID. You scope this role to a subscription, but you can scope it to a single resource group if you prefer. The name must be unique to the tenant.

Important

To use a custom role, you need a Microsoft Entra ID P1 license. For more information about licenses, see Microsoft Entra plans and pricing.

Optional permissions

If you're scoping CycleCloud to use a single resource group for each cluster, you can remove the following permissions from actions:

          "Microsoft.Resources/subscriptions/resourceGroups/write",
          "Microsoft.Resources/subscriptions/resourceGroups/delete",

If you're not using CycleCloud to assign managed identities to the VMs it creates within clusters, you can remove the following permissions from actions:

          "Microsoft.Authorization/*/read",
          "Microsoft.Authorization/roleAssignments/*",
          "Microsoft.Authorization/roleDefinitions/*",

Warning

Future versions of CycleCloud require the ability to assign managed identities to VMs. We don't recommend removing these permissions.

Creating the role

You can create a role from the role definitions by using the Azure CLI. Use this role to create a role definition within the Azure tenant. When the role exists in the tenant, assign the role to an identity with the proper scope.

The following example shows the basic flow using the Azure CLI.

# Create a custom role definition
az role definition create --role-definition role.json
# Create user identity
az identity create --name <name>
# Assign the custom role to the identity with proper scope
az role assignment create --role <CycleCloudRole> --assignee-object-id <identity-id> --scope <subscription>

Now the custom role is assigned and scoped to the identity. You can use it with a VM.

Assigning roles to cluster VMs with managed identity

Cluster nodes often need access to Azure resources. For example, many clusters require access to Azure Storage, Key Vault, or Azure Container Registries to run their workload. We strongly recommend using a User-Assigned Managed Identity to provide access credentials instead of passing secrets or credentials to the node through cluster configuration.

You can configure user-assigned managed identities on the cluster VMs by using the Azure.Identities node property. Set the Azure.Identities property to a comma-separated list of managed identity resource ID strings:

[cluster sample]
...
    [[node defaults]]
    ...
    Azure.Identities = $ManagedServiceIdentity
...

[parameters Required Settings]
...
  [[parameter ManagedServiceIdentity]]
  ParameterType = Azure.ManagedIdentity
  Label = MSI Identity
  Description = The resource ID of the Managed Service Identity to apply to the nodes
...