Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
This article describes how to create an Azure Red Hat OpenShift with hosted control planes (HCP) cluster by using the Azure CLI or a Bicep file.
The Bicep file defines all of the Azure resources required for the cluster, including the network infrastructure, managed identities, role assignments, cluster configuration, and node pool.
The CLI commands create all of the Azure resources required for the cluster, including the network infrastructure, managed identities, role assignments, cluster configuration, and node pool.
Prerequisites
Prepare your environment
Before you create the cluster, verify that your Azure environment meets the following requirements.
Azure CLI
Ensure you're using Azure CLI version 2.67.0 or higher.
Use az --version to check your installed version.
To install or upgrade, see Install Azure CLI.
ARO HCP CLI extension
Download the wheel file for the ARO HCP CLI extension.
Install the extension. Replace
<path-to-downloaded-extension.whl>with the path to the downloaded file.az extension add --source <path-to-downloaded-extension.whl>Verify that the extension is installed:
az aro hcp -h
Resource quota
Azure Red Hat OpenShift with hosted control planes requires at least 20 cores to create and run a cluster. The default Azure resource quota for a new Azure subscription doesn't meet this requirement. To request an increase in your resource limit, see Standard quota: Increase limits by VM series. Verify that you have enough quota for the VM family you choose from the supported worker node VM sizes.
To check the current quota for the default virtual machine family:
LOCATION=eastus
az vm list-usage -l $LOCATION \
--query "[?contains(name.value, 'standardDSv5Family')]" -o table
Permissions
You need Contributor and User Access Administrator permissions, or Owner permissions, on the resource group or subscription where you create the cluster. These permissions are required to create the resource group, virtual network, managed identities, and role assignments.
For more information, see Verify your permissions.
Register resource providers
Register the following resource providers in your Azure subscription:
Microsoft.RedHatOpenShiftMicrosoft.ComputeMicrosoft.StorageMicrosoft.Authorization
To check whether a resource provider is registered:
az provider list --query "[?namespace=='Microsoft.RedHatOpenShift'].registrationState" \
--output table
If you find any resource providers that aren't registered, register them.
az provider register --namespace Microsoft.RedHatOpenShift --wait
az provider register --namespace Microsoft.Compute --wait
az provider register --namespace Microsoft.Storage --wait
az provider register --namespace Microsoft.Authorization --wait
For more information about how to register resource providers, see Register the resource providers.
Set environment variables
Set the following environment variables to define the names of the resources that you create. Replace the placeholder values with your own.
# Azure resource location and subscription
LOCATION="<location>"
SUBSCRIPTION_ID="$(az account show --query id --output tsv)"
CUSTOMER_RG_NAME="<resource-group-name>"
# Network resources
CUSTOMER_NSG="<nsg-name>"
CUSTOMER_VNET_NAME="<vnet-name>"
CUSTOMER_VNET_SUBNET1="<worker-subnet-name>"
CUSTOMER_VNET_INTEGRATION_SUBNET_NAME="<vnet-integration-subnet-name>"
# Cluster and node pool
CLUSTER_NAME="<cluster-name>"
MANAGED_RESOURCE_GROUP="${CLUSTER_NAME}-managed-rg"
NP_NAME="<node-pool-name>"
CLUSTER_VERSION="<major.minor>" # Example: 4.22
NP_VERSION="<major.minor.patch>" # Example: 4.22.1
Note
For more information about cluster and node pool version requirements, review the supported version combinations.
Create a resource group
Create a resource group to hold the cluster resource, virtual network, and managed identities.
az group create \
--name "${CUSTOMER_RG_NAME}" \
--subscription "${SUBSCRIPTION_ID}" \
--location "${LOCATION}"
Create the Bicep file
Create a file named azuredeploy.bicep with the following content.
This Bicep template defines all of the resources required to deploy an Azure Red Hat OpenShift with hosted control planes cluster, including the network security group, virtual network, managed identities, role assignments, Key Vault, KMS encryption key, cluster, and node pool.
The template deploys a cluster with OVN-Kubernetes networking with default CIDR ranges, customer-managed etcd encryption, load balancer outbound connectivity, and a single node pool with two Standard_D8s_v3 worker nodes.
To customize the cluster or node pool configuration, see Bicep file reference.
@description('Network Security Group Name')
param customerNsgName string
@description('Virtual Network Name')
param customerVnetName string
@description('Subnet Name')
param customerVnetSubnetName string
@description('Virtual Network Integration Subnet Name')
param customerVirtualNetworkIntegrationSubnetName string
@description('Name of the cluster')
param clusterName string
@description('The cluster managed resource group name')
param managedResourceGroupName string
@description('The name of the node pool')
param nodePoolName string
@description('The OpenShift version for the cluster (X.Y format, e.g. 4.20)')
param clusterVersion string
@description('The OpenShift version for the node pool (X.Y.Z format, e.g. 4.20.8)')
param nodePoolVersion string
@description('Deploy ARO HCP with a private key vault')
param privateKeyVault bool
@description('API server visibility')
@allowed([
'Public'
'Private'
])
param apiVisibility string = 'Public'
@description('Default ingress type')
@allowed([
'Public'
'Private'
])
param ingressType string = 'Public'
@description('Cryptographic restrictions for kernel and userspace libraries (immutable after creation)')
@allowed(['None', 'FIPS'])
param cryptoRestrictions string = 'None'
var etcdEncryptionKeyName = 'etcd-data-kms-encryption-key'
var randomSuffix = toLower(uniqueString(clusterName))
var randomKeyVaultSuffix = toLower(uniqueString(resourceGroup().id))
var customerKeyVaultName string = 'cust-kv-${randomKeyVaultSuffix}'
var addressPrefix = '10.0.0.0/16'
var subnetPrefix = '10.0.0.0/24'
var virtualNetworkIntegrationSubnetPrefix = '10.0.1.0/24'
// Network Security Group
resource customerNsg 'Microsoft.Network/networkSecurityGroups@2023-05-01' = {
name: customerNsgName
location: resourceGroup().location
}
// Virtual network with worker subnet and VNet integration subnet
resource customerVnet 'Microsoft.Network/virtualNetworks@2023-05-01' = {
name: customerVnetName
location: resourceGroup().location
properties: {
addressSpace: {
addressPrefixes: [
addressPrefix
]
}
subnets: [
{
name: customerVnetSubnetName
properties: {
addressPrefix: subnetPrefix
networkSecurityGroup: {
id: customerNsg.id
}
}
}
{
name: customerVirtualNetworkIntegrationSubnetName
properties: {
addressPrefix: virtualNetworkIntegrationSubnetPrefix
networkSecurityGroup: {
id: customerNsg.id
}
delegations: [
{
name: 'aro-hcp-delegation'
properties: {
serviceName: 'Microsoft.RedHatOpenShift/hcpOpenShiftClusters'
}
}
]
}
}
]
}
}
resource subnet 'Microsoft.Network/virtualNetworks/subnets@2022-07-01' existing = {
name: customerVnetSubnetName
parent: customerVnet
}
resource vnetIntegrationSubnet 'Microsoft.Network/virtualNetworks/subnets@2022-07-01' existing = {
name: customerVirtualNetworkIntegrationSubnetName
parent: customerVnet
}
resource customerKeyVault 'Microsoft.KeyVault/vaults@2024-12-01-preview' = {
name: customerKeyVaultName
location: resourceGroup().location
properties: {
enableRbacAuthorization: true
tenantId: subscription().tenantId
publicNetworkAccess: privateKeyVault ? 'Disabled' : 'Enabled'
sku: {
family: 'A'
name: 'standard'
}
}
}
resource etcdEncryptionKey 'Microsoft.KeyVault/vaults/keys@2024-12-01-preview' = {
parent: customerKeyVault
name: 'etcd-data-kms-encryption-key'
properties: {
kty: 'RSA'
keySize: 2048
}
}
resource privateEndpointDnsZone 'Microsoft.Network/privateDnsZones@2020-06-01' = if (privateKeyVault) {
name: 'privatelink.vaultcore.azure.net'
location: 'global'
properties: {}
dependsOn: [
privateEndpoint
]
}
resource privateEndpoint 'Microsoft.Network/privateEndpoints@2023-09-01' = if (privateKeyVault) {
name: 'kv-private-endpoint'
properties: {
privateLinkServiceConnections: [
{
name: 'kv-private-endpoint'
properties: {
privateLinkServiceId: customerKeyVault.id
groupIds: ['vault']
}
}
]
subnet: {
id: customerVnet.properties.subnets[0].id
}
}
location: resourceGroup().location
}
resource privateEndpointDnsGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-09-01' = if (privateKeyVault) {
name: 'kv-private-ep-dns-group'
parent: privateEndpoint
properties: {
privateDnsZoneConfigs: [
{
name: 'config1'
properties: {
privateDnsZoneId: privateEndpointDnsZone.id
}
}
]
}
dependsOn: [
privateDnsZoneVnetLink
]
}
resource privateDnsZoneVnetLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = if (privateKeyVault) {
name: uniqueString('kv-private-dns-zone-link')
parent: privateEndpointDnsZone
location: 'global'
properties: {
registrationEnabled: false
virtualNetwork: {
id: customerVnet.id
}
}
}
//
// Control plane identities
//
// Reader
var readerRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'acdd72a7-3385-48ef-bd42-f606fba81ae7'
)
//
// Cluster API Azure managed identity
//
resource clusterApiAzureMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-cluster-api-azure-${randomSuffix}'
location: resourceGroup().location
}
// Azure Red Hat OpenShift with hosted control planes Cluster API Provider
var hcpClusterApiProviderRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'88366f10-ed47-4cc0-9fab-c8a06148393e'
)
resource hcpClusterApiProviderRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, clusterApiAzureMi.id, hcpClusterApiProviderRoleId, subnet.id)
scope: subnet
properties: {
principalId: clusterApiAzureMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpClusterApiProviderRoleId
}
}
resource hcpClusterApiProviderRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, clusterApiAzureMi.id, hcpClusterApiProviderRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: clusterApiAzureMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpClusterApiProviderRoleId
}
}
resource serviceManagedIdentityReaderOnClusterApiAzureMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, clusterApiAzureMi.id)
scope: clusterApiAzureMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// KMS managed identity
//
resource kmsMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-kms-${randomSuffix}'
location: resourceGroup().location
}
// Key Vault Crypto User
var keyVaultCryptoUserRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'12338af0-0e69-4776-bea7-57ae8d297424'
)
resource keyVaultCryptoUserToKeyVaultRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, kmsMi.id, keyVaultCryptoUserRoleId, customerKeyVault.id)
scope: customerKeyVault
properties: {
principalId: kmsMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: keyVaultCryptoUserRoleId
}
}
resource serviceManagedIdentityReaderOnKmsMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, kmsMi.id)
scope: kmsMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Control plane operator managed identity
//
resource controlPlaneMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-control-plane-${randomSuffix}'
location: resourceGroup().location
}
// Azure Red Hat OpenShift with hosted control planes Control Plane Operator
var hcpControlPlaneOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'fc0c873f-45e9-4d0d-a7d1-585aab30c6ed'
)
resource hcpControlPlaneOperatorVnetRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, controlPlaneMi.id, hcpControlPlaneOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: controlPlaneMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpControlPlaneOperatorRoleId
}
}
resource hcpControlPlaneOperatorNsgRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, controlPlaneMi.id, hcpControlPlaneOperatorRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: controlPlaneMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpControlPlaneOperatorRoleId
}
}
resource serviceManagedIdentityReaderOnControlPlaneMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, controlPlaneMi.id)
scope: controlPlaneMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Cloud controller manager managed identity
//
resource cloudControllerManagerMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-cloud-controller-manager-${randomSuffix}'
location: resourceGroup().location
}
// Azure Red Hat OpenShift Cloud Controller Manager
var cloudControllerManagerRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'a1f96423-95ce-4224-ab27-4e3dc72facd4'
)
resource cloudControllerManagerRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, cloudControllerManagerMi.id, cloudControllerManagerRoleId, subnet.id)
scope: subnet
properties: {
principalId: cloudControllerManagerMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: cloudControllerManagerRoleId
}
}
resource cloudControllerManagerRoleNsgAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, cloudControllerManagerMi.id, cloudControllerManagerRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: cloudControllerManagerMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: cloudControllerManagerRoleId
}
}
resource cloudControllerManagerRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, cloudControllerManagerMi.id, cloudControllerManagerRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: cloudControllerManagerMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: cloudControllerManagerRoleId
}
}
resource serviceManagedIdentityReaderOnCloudControllerManagerMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, cloudControllerManagerMi.id)
scope: cloudControllerManagerMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Ingress managed identity
//
resource ingressMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-ingress-${randomSuffix}'
location: resourceGroup().location
}
// Azure Red Hat OpenShift Cluster Ingress Operator
var ingressOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'0336e1d3-7a87-462b-b6db-342b63f7802c'
)
// Azure Red Hat OpenShift Image Registry Operator
var imageRegistryOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'8b32b316-c2f5-4ddf-b05b-83dacd2d08b5'
)
resource ingressOperatorRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, ingressMi.id, ingressOperatorRoleId, subnet.id)
scope: subnet
properties: {
principalId: ingressMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: ingressOperatorRoleId
}
}
resource ingressOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, ingressMi.id, ingressOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: ingressMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: ingressOperatorRoleId
}
}
resource serviceManagedIdentityReaderOnIngressMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, ingressMi.id)
scope: ingressMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Disk CSI driver managed identity
//
resource diskCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-disk-csi-driver-${randomSuffix}'
location: resourceGroup().location
}
resource serviceManagedIdentityReaderOnDiskCsiDriverMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, diskCsiDriverMi.id)
scope: diskCsiDriverMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// File CSI driver managed identity
//
resource fileCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-file-csi-driver-${randomSuffix}'
location: resourceGroup().location
}
// Azure Red Hat OpenShift File Storage Operator
var fileStorageOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'0d7aedc0-15fd-4a67-a412-efad370c947e'
)
resource fileStorageOperatorRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, fileCsiDriverMi.id, fileStorageOperatorRoleId, subnet.id)
scope: subnet
properties: {
principalId: fileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource fileStorageOperatorRoleNsgAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, fileCsiDriverMi.id, fileStorageOperatorRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: fileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource fileStorageOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, fileCsiDriverMi.id, fileStorageOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: fileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource serviceManagedIdentityReaderOnFileCsiDriverMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, fileCsiDriverMi.id)
scope: fileCsiDriverMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Image registry managed identity
//
resource imageRegistryMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-image-registry-${randomSuffix}'
location: resourceGroup().location
}
resource serviceManagedIdentityReaderOnImageRegistryMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, imageRegistryMi.id)
scope: imageRegistryMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
resource imageRegistryOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, imageRegistryMi.id, imageRegistryOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: imageRegistryMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: imageRegistryOperatorRoleId
}
}
//
// Cloud network config managed identity
//
resource cloudNetworkConfigMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-cp-cloud-network-config-${randomSuffix}'
location: resourceGroup().location
}
// Azure Red Hat OpenShift Network Operator
var networkOperatorRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'be7a6435-15ae-4171-8f30-4a343eff9e8f'
)
resource networkOperatorRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, cloudNetworkConfigMi.id, networkOperatorRoleId, subnet.id)
scope: subnet
properties: {
principalId: cloudNetworkConfigMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: networkOperatorRoleId
}
}
resource networkOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, cloudNetworkConfigMi.id, networkOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: cloudNetworkConfigMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: networkOperatorRoleId
}
}
resource serviceManagedIdentityReaderOnCloudNetworkMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, cloudNetworkConfigMi.id)
scope: cloudNetworkConfigMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: readerRoleId
}
}
//
// Data plane identities
//
// Azure Red Hat OpenShift Federated Credential
var federatedCredentialsRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'ef318e2a-8334-4a05-9e4a-295a196c6a6e'
)
resource dpDiskCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-dp-disk-csi-driver-${randomSuffix}'
location: resourceGroup().location
}
resource dpDiskCsiDriverMiFederatedCredentialsRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpDiskCsiDriverMi.id, federatedCredentialsRoleId)
scope: dpDiskCsiDriverMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: federatedCredentialsRoleId
}
}
resource dpFileCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-dp-file-csi-driver-${randomSuffix}'
location: resourceGroup().location
}
resource dpFileCsiDriverMiFederatedCredentialsRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpFileCsiDriverMi.id, federatedCredentialsRoleId)
scope: dpFileCsiDriverMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: federatedCredentialsRoleId
}
}
resource dpFileCsiDriverFileStorageOperatorRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpFileCsiDriverMi.id, fileStorageOperatorRoleId, subnet.id)
scope: subnet
properties: {
principalId: dpFileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource dpFileCsiDriverFileStorageOperatorRoleNsgAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpFileCsiDriverMi.id, fileStorageOperatorRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: dpFileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource dpFileCsiDriverFileStorageOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpFileCsiDriverMi.id, fileStorageOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: dpFileCsiDriverMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: fileStorageOperatorRoleId
}
}
resource dpImageRegistryMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-dp-image-registry-${randomSuffix}'
location: resourceGroup().location
}
resource dpImageRegistryMiFederatedCredentialsRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpImageRegistryMi.id, federatedCredentialsRoleId)
scope: dpImageRegistryMi
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: federatedCredentialsRoleId
}
}
resource dpImageRegistryOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, dpImageRegistryMi.id, imageRegistryOperatorRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: dpImageRegistryMi.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: imageRegistryOperatorRoleId
}
}
//
// Service managed identity
//
resource serviceManagedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
name: '${clusterName}-service-managed-identity-${randomSuffix}'
location: resourceGroup().location
}
// Azure Red Hat OpenShift with hosted control planes Service Managed Identity
var hcpServiceManagedIdentityRoleId = subscriptionResourceId(
'Microsoft.Authorization/roleDefinitions',
'c0ff367d-66d8-445e-917c-583feb0ef0d4'
)
resource serviceManagedIdentityRoleAssignmentVnet 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, hcpServiceManagedIdentityRoleId, customerVnet.id)
scope: customerVnet
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpServiceManagedIdentityRoleId
}
}
resource serviceManagedIdentityRoleAssignmentNSG 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
name: guid(resourceGroup().id, serviceManagedIdentity.id, hcpServiceManagedIdentityRoleId, customerNsg.id)
scope: customerNsg
properties: {
principalId: serviceManagedIdentity.properties.principalId
principalType: 'ServicePrincipal'
roleDefinitionId: hcpServiceManagedIdentityRoleId
}
}
// Cluster resource
resource hcp 'Microsoft.RedHatOpenShift/hcpOpenShiftClusters@2026-09-01-preview' = {
name: clusterName
location: resourceGroup().location
properties: {
version: {
id: clusterVersion
channelGroup: 'stable'
}
dns: {}
network: {
networkType: 'OVNKubernetes'
podCidr: '10.128.0.0/14'
serviceCidr: '172.30.0.0/16'
machineCidr: '10.0.0.0/16'
hostPrefix: 23
}
etcd: {
dataEncryption: {
keyManagementMode: 'CustomerManaged'
customerManaged: {
encryptionType: 'KMS'
kms: {
activeKey: {
name: etcdEncryptionKeyName
version: last(split(etcdEncryptionKey.properties.keyUriWithVersion, '/'))
}
vaultName: customerKeyVaultName
visibility: privateKeyVault ? 'Private' : 'Public'
}
}
}
}
api: {
visibility: apiVisibility
}
ingress: {
type: ingressType
}
cryptoRestrictions: cryptoRestrictions
clusterImageRegistry: {
state: 'Enabled'
}
platform: {
managedResourceGroup: managedResourceGroupName
subnetId: subnet.id
vnetIntegrationSubnetId: vnetIntegrationSubnet.id
outboundType: 'LoadBalancer'
networkSecurityGroupId: customerNsg.id
operatorsAuthentication: {
userAssignedIdentities: {
controlPlaneOperators: {
'cluster-api-azure': clusterApiAzureMi.id
'control-plane': controlPlaneMi.id
'cloud-controller-manager': cloudControllerManagerMi.id
ingress: ingressMi.id
'disk-csi-driver': diskCsiDriverMi.id
'file-csi-driver': fileCsiDriverMi.id
'image-registry': imageRegistryMi.id
'cloud-network-config': cloudNetworkConfigMi.id
kms: kmsMi.id
}
dataPlaneOperators: {
'disk-csi-driver': dpDiskCsiDriverMi.id
'file-csi-driver': dpFileCsiDriverMi.id
'image-registry': dpImageRegistryMi.id
}
serviceManagedIdentity: serviceManagedIdentity.id
}
}
}
}
identity: {
type: 'UserAssigned'
userAssignedIdentities: {
'${serviceManagedIdentity.id}': {}
'${clusterApiAzureMi.id}': {}
'${controlPlaneMi.id}': {}
'${cloudControllerManagerMi.id}': {}
'${ingressMi.id}': {}
'${diskCsiDriverMi.id}': {}
'${fileCsiDriverMi.id}': {}
'${imageRegistryMi.id}': {}
'${cloudNetworkConfigMi.id}': {}
'${kmsMi.id}': {}
}
}
dependsOn: [
hcpClusterApiProviderRoleSubnetAssignment
hcpClusterApiProviderRoleVnetAssignment
keyVaultCryptoUserToKeyVaultRoleAssignment
hcpControlPlaneOperatorVnetRoleAssignment
hcpControlPlaneOperatorNsgRoleAssignment
cloudControllerManagerRoleSubnetAssignment
cloudControllerManagerRoleNsgAssignment
cloudControllerManagerRoleVnetAssignment
ingressOperatorRoleSubnetAssignment
ingressOperatorRoleVnetAssignment
fileStorageOperatorRoleSubnetAssignment
fileStorageOperatorRoleNsgAssignment
fileStorageOperatorRoleVnetAssignment
networkOperatorRoleSubnetAssignment
networkOperatorRoleVnetAssignment
dpDiskCsiDriverMiFederatedCredentialsRoleAssignment
dpFileCsiDriverMiFederatedCredentialsRoleAssignment
dpImageRegistryMiFederatedCredentialsRoleAssignment
serviceManagedIdentityRoleAssignmentVnet
serviceManagedIdentityRoleAssignmentNSG
dpFileCsiDriverFileStorageOperatorRoleSubnetAssignment
dpFileCsiDriverFileStorageOperatorRoleNsgAssignment
dpFileCsiDriverFileStorageOperatorRoleVnetAssignment
imageRegistryOperatorRoleVnetAssignment
dpImageRegistryOperatorRoleVnetAssignment
serviceManagedIdentityReaderOnControlPlaneMi
serviceManagedIdentityReaderOnCloudControllerManagerMi
serviceManagedIdentityReaderOnIngressMi
serviceManagedIdentityReaderOnDiskCsiDriverMi
serviceManagedIdentityReaderOnFileCsiDriverMi
serviceManagedIdentityReaderOnImageRegistryMi
serviceManagedIdentityReaderOnCloudNetworkMi
serviceManagedIdentityReaderOnClusterApiAzureMi
serviceManagedIdentityReaderOnKmsMi
]
}
// Node pool resource
resource nodepool 'Microsoft.RedHatOpenShift/hcpOpenShiftClusters/nodePools@2026-09-01-preview' = {
parent: hcp
name: nodePoolName
location: resourceGroup().location
properties: {
version: {
id: nodePoolVersion
channelGroup: 'stable'
}
platform: {
subnetId: hcp.properties.platform.subnetId
vmSize: 'Standard_D8s_v3'
osDisk: {
sizeGiB: 64
diskStorageAccountType: 'StandardSSD_LRS'
}
}
replicas: 2
}
}
Deploy the Bicep file
Deploy the Bicep template to create the cluster and all required resources.
az deployment group create \
--name 'aro-hcp' \
--subscription "${SUBSCRIPTION_ID}" \
--resource-group "${CUSTOMER_RG_NAME}" \
--template-file azuredeploy.bicep \
--parameters \
customerNsgName="${CUSTOMER_NSG}" \
customerVnetName="${CUSTOMER_VNET_NAME}" \
customerVirtualNetworkIntegrationSubnetName="${CUSTOMER_VNET_INTEGRATION_SUBNET_NAME}" \
customerVnetSubnetName="${CUSTOMER_VNET_SUBNET1}" \
clusterName="${CLUSTER_NAME}" \
managedResourceGroupName="${MANAGED_RESOURCE_GROUP}" \
nodePoolName="${NP_NAME}" \
clusterVersion="${CLUSTER_VERSION}" \
nodePoolVersion="${NP_VERSION}" \
apiVisibility="<Public-or-Private>" \
ingressType="<Public-or-Private>" \
privateKeyVault=<true-or-false> \
cryptoRestrictions="<None-or-FIPS>"
Set apiVisibility to Public for an internet-accessible API server, or Private for an API server reachable only through a private network connection.
Set ingressType to Public for internet-accessible application routes, or Private for application routes reachable only through a private network connection.
Set privateKeyVault to true to restrict access to the Key Vault to a private endpoint, or false to allow public access. If you set privateKeyVault to true, the Bicep template disables public Key Vault access, deploys the private endpoint resources, and sets vault visibility to Private on your cluster.
Set cryptoRestrictions to FIPS to restrict the cluster to FIPS-validated cryptographic modules, or None (default) to deploy without FIPS restrictions. This setting is immutable after cluster creation.
Note
The control plane is always hosted on a FIPS-enabled cluster, regardless of this setting. The cryptoRestrictions parameter controls whether the worker node pools use FIPS-validated cryptographic modules.
The deployment typically takes 15 to 20 minutes to complete.
Verify the deployment
After the deployment finishes, verify that the cluster and node pool were created successfully.
Verify that the cluster was created successfully.
az aro hcp cluster show \ --name "${CLUSTER_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query "properties.provisioningState" \ --output tsvWhen the cluster is ready, the output is
Succeeded.Verify that the node pool was created successfully.
az aro hcp cluster nodepool show \ --cluster-name "${CLUSTER_NAME}" \ --name "${NP_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query "properties.provisioningState" \ --output tsvWhen the node pool is ready, the output is
Succeeded.
Create the network infrastructure
Create a network security group, a virtual network with a worker subnet, and a VNet integration subnet with delegation.
Create the network security group.
az network nsg create \ --name "${CUSTOMER_NSG}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --location "${LOCATION}"Create the virtual network with the worker subnet.
az network vnet create \ --name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --location "${LOCATION}" \ --address-prefixes 10.0.0.0/16 \ --subnet-name "${CUSTOMER_VNET_SUBNET1}" \ --subnet-prefixes 10.0.0.0/24 \ --nsg "${CUSTOMER_NSG}"Create the VNet integration subnet with the required delegation.
az network vnet subnet create \ --name "${CUSTOMER_VNET_INTEGRATION_SUBNET_NAME}" \ --vnet-name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --address-prefixes 10.0.1.0/24 \ --network-security-group "${CUSTOMER_NSG}" \ --delegations Microsoft.RedHatOpenShift/hcpOpenShiftClustersGet the resource IDs for the networking resources.
NSG_ID=$(az network nsg show \ --name "${CUSTOMER_NSG}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query id --output tsv) SUBNET_ID=$(az network vnet subnet show \ --name "${CUSTOMER_VNET_SUBNET1}" \ --vnet-name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query id --output tsv) VNET_ID=$(az network vnet show \ --name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query id --output tsv) VNET_INTEGRATION_SUBNET_ID=$(az network vnet subnet show \ --name "${CUSTOMER_VNET_INTEGRATION_SUBNET_NAME}" \ --vnet-name "${CUSTOMER_VNET_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --query id --output tsv)
Create the managed identities
Create the user-assigned managed identities for the cluster operators and the service managed identity. For a description of each identity and its purpose, see Required managed identities and role assignments.
# Control plane operator identities
az identity create --name "${CLUSTER_NAME}-cp-cluster-api-azure" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-control-plane" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-cloud-controller-manager" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-ingress" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-image-registry" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-cloud-network-config" --resource-group "${CUSTOMER_RG_NAME}"
# Data plane operator identities
az identity create --name "${CLUSTER_NAME}-dp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-dp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-dp-image-registry" --resource-group "${CUSTOMER_RG_NAME}"
# Service managed identity
az identity create --name "${CLUSTER_NAME}-service-managed-identity" --resource-group "${CUSTOMER_RG_NAME}"
Get the resource IDs and principal IDs for the managed identities.
# Control plane identity resource IDs and principal IDs
CLUSTER_API_AZURE_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-cluster-api-azure" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CLUSTER_API_AZURE_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-cluster-api-azure" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
CONTROL_PLANE_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-control-plane" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CONTROL_PLANE_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-control-plane" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
CLOUD_CONTROLLER_MANAGER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-controller-manager" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CLOUD_CONTROLLER_MANAGER_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-controller-manager" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
INGRESS_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-ingress" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
INGRESS_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-ingress" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
DISK_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
FILE_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
FILE_CSI_DRIVER_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
IMAGE_REGISTRY_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
IMAGE_REGISTRY_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
CLOUD_NETWORK_CONFIG_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-network-config" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CLOUD_NETWORK_CONFIG_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-network-config" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
# Data plane identity resource IDs and principal IDs
DP_DISK_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-dp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
DP_FILE_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-dp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
DP_FILE_CSI_DRIVER_MI_PID=$(az identity show --name "${CLUSTER_NAME}-dp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
DP_IMAGE_REGISTRY_MI_ID=$(az identity show --name "${CLUSTER_NAME}-dp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
DP_IMAGE_REGISTRY_MI_PID=$(az identity show --name "${CLUSTER_NAME}-dp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
# Service managed identity resource ID and principal ID
SERVICE_MI_ID=$(az identity show --name "${CLUSTER_NAME}-service-managed-identity" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
SERVICE_MI_PID=$(az identity show --name "${CLUSTER_NAME}-service-managed-identity" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
Create the role assignments
Assign the required Azure roles to each managed identity. For a detailed description of each identity and its required role assignments, see Required managed identities and role assignments.
Set the role definition GUIDs.
READER_ROLE="acdd72a7-3385-48ef-bd42-f606fba81ae7" HCP_CLUSTER_API_PROVIDER_ROLE="88366f10-ed47-4cc0-9fab-c8a06148393e" HCP_CONTROL_PLANE_OPERATOR_ROLE="fc0c873f-45e9-4d0d-a7d1-585aab30c6ed" CLOUD_CONTROLLER_MANAGER_ROLE="a1f96423-95ce-4224-ab27-4e3dc72facd4" INGRESS_OPERATOR_ROLE="0336e1d3-7a87-462b-b6db-342b63f7802c" FILE_STORAGE_OPERATOR_ROLE="0d7aedc0-15fd-4a67-a412-efad370c947e" NETWORK_OPERATOR_ROLE="be7a6435-15ae-4171-8f30-4a343eff9e8f" FEDERATED_CREDENTIAL_ROLE="ef318e2a-8334-4a05-9e4a-295a196c6a6e" HCP_SERVICE_MI_ROLE="c0ff367d-66d8-445e-917c-583feb0ef0d4" IMAGE_REGISTRY_OPERATOR_ROLE="8b32b316-c2f5-4ddf-b05b-83dacd2d08b5"Assign control plane operator roles.
# Cluster API Azure → ARO HCP Cluster API Provider on worker subnet az role assignment create --assignee-object-id "${CLUSTER_API_AZURE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CLUSTER_API_PROVIDER_ROLE}" --scope "${SUBNET_ID}" az role assignment create --assignee-object-id "${CLUSTER_API_AZURE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CLUSTER_API_PROVIDER_ROLE}" --scope "${VNET_ID}" # Control Plane → ARO HCP Control Plane Operator on VNet and NSG az role assignment create --assignee-object-id "${CONTROL_PLANE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CONTROL_PLANE_OPERATOR_ROLE}" --scope "${VNET_ID}" az role assignment create --assignee-object-id "${CONTROL_PLANE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CONTROL_PLANE_OPERATOR_ROLE}" --scope "${NSG_ID}" # Cloud Controller Manager → ARO Cloud Controller Manager on worker subnet and NSG az role assignment create --assignee-object-id "${CLOUD_CONTROLLER_MANAGER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${CLOUD_CONTROLLER_MANAGER_ROLE}" --scope "${SUBNET_ID}" az role assignment create --assignee-object-id "${CLOUD_CONTROLLER_MANAGER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${CLOUD_CONTROLLER_MANAGER_ROLE}" --scope "${NSG_ID}" az role assignment create --assignee-object-id "${CLOUD_CONTROLLER_MANAGER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${CLOUD_CONTROLLER_MANAGER_ROLE}" --scope "${VNET_ID}" # Ingress → ARO Cluster Ingress Operator on worker subnet az role assignment create --assignee-object-id "${INGRESS_MI_PID}" --assignee-principal-type ServicePrincipal --role "${INGRESS_OPERATOR_ROLE}" --scope "${SUBNET_ID}" az role assignment create --assignee-object-id "${INGRESS_MI_PID}" --assignee-principal-type ServicePrincipal --role "${INGRESS_OPERATOR_ROLE}" --scope "${VNET_ID}" # File CSI Driver → ARO File Storage Operator on worker subnet and NSG az role assignment create --assignee-object-id "${FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${SUBNET_ID}" az role assignment create --assignee-object-id "${FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${NSG_ID}" az role assignment create --assignee-object-id "${FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${VNET_ID}" # Cloud Network Config → ARO Network Operator on worker subnet and VNet az role assignment create --assignee-object-id "${CLOUD_NETWORK_CONFIG_MI_PID}" --assignee-principal-type ServicePrincipal --role "${NETWORK_OPERATOR_ROLE}" --scope "${SUBNET_ID}" az role assignment create --assignee-object-id "${CLOUD_NETWORK_CONFIG_MI_PID}" --assignee-principal-type ServicePrincipal --role "${NETWORK_OPERATOR_ROLE}" --scope "${VNET_ID}" # Image Registry → ARO Image Registry Operator on VNet az role assignment create --assignee-object-id "${IMAGE_REGISTRY_MI_PID}" --assignee-principal-type ServicePrincipal --role "${IMAGE_REGISTRY_OPERATOR_ROLE}" --scope "${VNET_ID}"Assign service managed identity roles.
# Service MI → ARO HCP Service Managed Identity on VNet and NSG az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_SERVICE_MI_ROLE}" --scope "${VNET_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_SERVICE_MI_ROLE}" --scope "${NSG_ID}" # Service MI → Reader on each control plane identity az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CLUSTER_API_AZURE_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CONTROL_PLANE_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CLOUD_CONTROLLER_MANAGER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${INGRESS_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${DISK_CSI_DRIVER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${FILE_CSI_DRIVER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${IMAGE_REGISTRY_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CLOUD_NETWORK_CONFIG_MI_ID}"Assign data plane operator roles.
# Service MI → ARO Federated Credential on each data plane identity az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FEDERATED_CREDENTIAL_ROLE}" --scope "${DP_DISK_CSI_DRIVER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FEDERATED_CREDENTIAL_ROLE}" --scope "${DP_FILE_CSI_DRIVER_MI_ID}" az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FEDERATED_CREDENTIAL_ROLE}" --scope "${DP_IMAGE_REGISTRY_MI_ID}" # Data Plane File CSI Driver → ARO File Storage Operator on worker subnet and NSG az role assignment create --assignee-object-id "${DP_FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${SUBNET_ID}" az role assignment create --assignee-object-id "${DP_FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${NSG_ID}" az role assignment create --assignee-object-id "${DP_FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${VNET_ID}" # Data Plane Image Registry → ARO Image Registry Operator on VNet az role assignment create --assignee-object-id "${DP_IMAGE_REGISTRY_MI_PID}" --assignee-principal-type ServicePrincipal --role "${IMAGE_REGISTRY_OPERATOR_ROLE}" --scope "${VNET_ID}"
Set up customer-managed etcd encryption
Azure Red Hat OpenShift with hosted control planes encrypts etcd data by using customer-managed keys.
Create the KMS managed identity
Set environment variables for the Key Vault and KMS key.
KEYVAULT_NAME="<key-vault-name>" KMS_KEY_NAME="etcd-kms-key"Create the KMS managed identity.
az identity create --name "${CLUSTER_NAME}-cp-kms" --resource-group "${CUSTOMER_RG_NAME}"Get the resource ID and principal ID for the KMS managed identity.
KMS_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-kms" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv) KMS_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-kms" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)
Create the KMS role assignments
Assign the service managed identity the Reader role on the KMS managed identity.
az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${KMS_MI_ID}"
Create the Key Vault and KMS key
Create an Azure Key Vault with RBAC authorization and an RSA 2048-bit encryption key for customer-managed etcd encryption.
Create the Key Vault.
az keyvault create \ --name "${KEYVAULT_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --location "${LOCATION}" \ --enable-rbac-authorization trueAssign yourself the Key Vault Crypto Officer user role. This role grants you the permissions to create the KMS encryption key.
az role assignment create \ --role "Key Vault Crypto Officer" \ --assignee "$(az account show --query user.name -o tsv)" \ --scope "$(az keyvault show --name "${KEYVAULT_NAME}" --resource-group "${CUSTOMER_RG_NAME}" --query id -o tsv)"Create the KMS encryption key.
az keyvault key create \ --vault-name "${KEYVAULT_NAME}" \ --name "${KMS_KEY_NAME}" \ --kty RSA \ --size 2048Get the key version.
KMS_KEY_VERSION=$(az keyvault key show \ --vault-name "${KEYVAULT_NAME}" \ --name "${KMS_KEY_NAME}" \ --query "key.kid" --output tsv | rev | cut -d'/' -f1 | rev)Assign the Key Vault Crypto User role to the KMS managed identity on the Key Vault.
KEY_VAULT_CRYPTO_USER_ROLE="12338af0-0e69-4776-bea7-57ae8d297424" KEYVAULT_ID=$(az keyvault show --name "${KEYVAULT_NAME}" --query id --output tsv) az role assignment create \ --assignee-object-id "${KMS_MI_PID}" \ --assignee-principal-type ServicePrincipal \ --role "${KEY_VAULT_CRYPTO_USER_ROLE}" \ --scope "${KEYVAULT_ID}"
(Optional) Restrict Key Vault access to a private endpoint
If you want to restrict Key Vault access to a private endpoint, create the following resources. If you don't need a private Key Vault, skip to Create the cluster.
Important
The steps in this section configure private access in Azure Key Vault only. When you create your Azure Red Hat OpenShift with hosted control planes cluster, also set --vault-visibility Private. Both settings must match.
Disable public network access on the Key Vault.
az keyvault update \ --name "${KEYVAULT_NAME}" \ --resource-group "${CUSTOMER_RG_NAME}" \ --public-network-access DisabledCreate a private endpoint for the Key Vault.
az network private-endpoint create \ --name "kv-private-endpoint" \ --resource-group "${CUSTOMER_RG_NAME}" \ --vnet-name "${CUSTOMER_VNET_NAME}" \ --subnet "${CUSTOMER_VNET_SUBNET1}" \ --private-connection-resource-id "${KEYVAULT_ID}" \ --group-ids vault \ --connection-name "kv-private-endpoint"Create a private DNS zone and link it to the virtual network.
az network private-dns zone create \ --name "privatelink.vaultcore.azure.net" \ --resource-group "${CUSTOMER_RG_NAME}" az network private-dns link vnet create \ --name "kv-private-dns-zone-link" \ --resource-group "${CUSTOMER_RG_NAME}" \ --zone-name "privatelink.vaultcore.azure.net" \ --virtual-network "${CUSTOMER_VNET_NAME}" \ --registration-enabled falseCreate a DNS zone group for the private endpoint.
az network private-endpoint dns-zone-group create \ --name "kv-private-ep-dns-group" \ --resource-group "${CUSTOMER_RG_NAME}" \ --endpoint-name "kv-private-endpoint" \ --private-dns-zone "privatelink.vaultcore.azure.net" \ --zone-name "config1"
Create the cluster
Create the Azure Red Hat OpenShift with hosted control planes cluster by using the network infrastructure, managed identities, and role assignments you configured in the previous steps.
Note
For information about all customizable cluster parameters, see the command help (az aro hcp cluster create -h).
Create the cluster with the following command:
az aro hcp cluster create \
--name "${CLUSTER_NAME}" \
--resource-group "${CUSTOMER_RG_NAME}" \
--location "${LOCATION}" \
--version "${CLUSTER_VERSION}" \
--channel-group stable \
--subnet-id "${SUBNET_ID}" \
--vnet-integration-subnet-id "${VNET_INTEGRATION_SUBNET_ID}" \
--nsg "${NSG_ID}" \
--managed-resource-group-name "${MANAGED_RESOURCE_GROUP}" \
--key-management-mode CustomerManaged \
--etcd-encryption-type KMS \
--kms-vault-name "${KEYVAULT_NAME}" \
--vault-visibility <Public-or-Private> \
--kms-active-key "{name:${KMS_KEY_NAME},version:${KMS_KEY_VERSION}}" \
--api-visibility <Public-or-Private> \
--ingress-visibility <Public-or-Private> \
--user-assigned-identities "{${SERVICE_MI_ID}:{},${CLUSTER_API_AZURE_MI_ID}:{},${CONTROL_PLANE_MI_ID}:{},${CLOUD_CONTROLLER_MANAGER_MI_ID}:{},${INGRESS_MI_ID}:{},${DISK_CSI_DRIVER_MI_ID}:{},${FILE_CSI_DRIVER_MI_ID}:{},${IMAGE_REGISTRY_MI_ID}:{},${CLOUD_NETWORK_CONFIG_MI_ID}:{},${KMS_MI_ID}:{}}" \
--operators-authentication "{user-assigned-identities:{control-plane-operators:{cluster-api-azure:${CLUSTER_API_AZURE_MI_ID},control-plane:${CONTROL_PLANE_MI_ID},cloud-controller-manager:${CLOUD_CONTROLLER_MANAGER_MI_ID},ingress:${INGRESS_MI_ID},disk-csi-driver:${DISK_CSI_DRIVER_MI_ID},file-csi-driver:${FILE_CSI_DRIVER_MI_ID},image-registry:${IMAGE_REGISTRY_MI_ID},cloud-network-config:${CLOUD_NETWORK_CONFIG_MI_ID},kms:${KMS_MI_ID}},data-plane-operators:{disk-csi-driver:${DP_DISK_CSI_DRIVER_MI_ID},file-csi-driver:${DP_FILE_CSI_DRIVER_MI_ID},image-registry:${DP_IMAGE_REGISTRY_MI_ID}},service-managed-identity:${SERVICE_MI_ID}}}"
If you restricted the Key Vault to a private endpoint, set --vault-visibility to Private instead of Public so your cluster setting matches the Azure Key Vault network configuration.
Set --api-visibility to Public for an internet-accessible API server, or Private for an API server reachable only through a private network connection.
Set --ingress-visibility to Public for internet-accessible application routes, or Private for application routes reachable only through a private network connection.
The deployment typically takes 15 to 20 minutes to complete.
Verify that the cluster was created successfully.
az aro hcp cluster show \
--name "${CLUSTER_NAME}" \
--resource-group "${CUSTOMER_RG_NAME}" \
--query "properties.provisioningState" \
--output tsv
When the cluster is ready, the output is Succeeded.
Create the node pool
Create a node pool to add worker compute capacity to your cluster.
The following command creates a node pool with two Standard_D8s_v3 worker nodes.
az aro hcp cluster nodepool create \
--cluster-name "${CLUSTER_NAME}" \
--name "${NP_NAME}" \
--resource-group "${CUSTOMER_RG_NAME}" \
--replicas 2 \
--vm-size Standard_D8s_v3 \
--version "${NP_VERSION}" \
--channel-group stable
Verify that the node pool was created successfully.
az aro hcp cluster nodepool show \
--cluster-name "${CLUSTER_NAME}" \
--name "${NP_NAME}" \
--resource-group "${CUSTOMER_RG_NAME}" \
--query "properties.provisioningState" \
--output tsv
When the node pool is ready, the output is Succeeded.
Bicep file reference
The example Bicep file deploys a cluster with default configuration values. This section describes each resource and property in the Bicep file that you can customize before deploying.
Networking resources
The Bicep file defines a network security group (NSG), a virtual network with a worker subnet and a VNet integration subnet, and the subnet delegation for the VNet integration subnet. For subnet sizing and placement requirements, see Plan your cluster network.
Managed identities and role assignments
The Bicep file defines a user-assigned managed identity for each OpenShift cluster Operator, and role assignments that grant each identity the permissions it needs. For a detailed description of each identity and its required role assignments, see Required managed identities and role assignments.
Etcd encryption resources
The Bicep file includes the following resources for customer-managed etcd encryption:
- KMS managed identity - A user-assigned managed identity to access the KMS key. This identity requires:
- A Key Vault Crypto User role assignment scoped to the Azure Key Vault
- A Reader role assignment that grants the service managed identity read access over the KMS managed identity
- Azure Key Vault - An Azure Key Vault with RBAC authorization enabled
- KMS encryption key - An RSA 2048 key created in the Key Vault
If you set the privateKeyVault parameter to true, the Bicep file also deploys the following resources to restrict access to the Key Vault to a private endpoint:
- A private endpoint for the Key Vault
- A private DNS zone (
privatelink.vaultcore.azure.net) - A private DNS zone group that associates the private endpoint with the DNS zone
- A virtual network link that connects the private DNS zone to the cluster virtual network
Cluster resource properties
The cluster resource (Microsoft.RedHatOpenShift/hcpOpenShiftClusters) defines the configuration for your cluster.
For more information about each configuration option, see Choose your permanent cluster settings.
| Property | Required | Description |
|---|---|---|
version.id |
Yes | The OpenShift minor version to install. For example, 4.20. To list available versions, see List available OpenShift versions. |
version.channelGroup |
Yes | The OpenShift update channel. The only supported option is stable. |
dns.baseDomainPrefix |
No | A custom prefix for the cluster's base domain name. Maximum length of 15 characters. Must start with a lowercase letter, and can contain only lowercase letters, numbers, and hyphens (-). |
network.networkType |
No | The Container Network Interface (CNI) plugin. Valid values are OVNKubernetes (default) or Other. |
network.podCidr |
No | The IP address range for pods. Default: 10.128.0.0/14. |
network.serviceCidr |
No | The IP address range for services. Default: 172.30.0.0/16. |
network.machineCidr |
No | The IP address range for compute nodes. Default: 10.0.0.0/16. |
network.hostPrefix |
No | The subnet prefix length allocated to each node for its pods. Default: 23. |
api.visibility |
No | The visibility of the API server. Valid values are Public (default) or Private. This setting is immutable. |
api.authorizedCidrs |
No | A list of authorized CIDR blocks (IPv4 only) permitted to access the API server. Maximum of 500 entries. |
ingress.type |
No | The type of the default cluster ingress. Valid values are Public (default) or Private. This setting is immutable. |
cryptoRestrictions |
No | Cryptographic restrictions for kernel and userspace libraries. Valid values are None (default) or FIPS. When set to FIPS, all cryptographic operations use FIPS-validated modules and all node pools inherit the FIPS configuration. This setting is immutable after cluster creation. |
etcd.dataEncryption.keyManagementMode |
Yes | The encryption key management mode. The only supported value is CustomerManaged. |
etcd.dataEncryption.customerManaged.encryptionType |
Yes | The encryption type. The only supported value is KMS. |
etcd.dataEncryption.customerManaged.kms.vaultName |
Yes | The name of the Azure Key Vault. |
etcd.dataEncryption.customerManaged.kms.visibility |
Yes | How your Azure Red Hat OpenShift with hosted control planes cluster reaches the Key Vault. Valid values are Public or Private. If you set this to Private, you must also disable public network access on the Key Vault and create a private endpoint in the cluster virtual network. |
etcd.dataEncryption.customerManaged.kms.activeKey.name |
Yes | The name of the encryption key. |
etcd.dataEncryption.customerManaged.kms.activeKey.version |
Yes | The version of the encryption key. |
clusterImageRegistry.state |
No | The state of the internal image registry. Valid values are Enabled (default) or Disabled. |
platform.managedResourceGroup |
No | The name for the managed resource group. Maximum length of 90 characters. |
platform.subnetId |
Yes | The Azure resource ID of the worker subnet. |
platform.vnetIntegrationSubnetId |
Yes | The Azure resource ID of the VNet integration subnet. |
platform.outboundType |
No | The outbound connectivity model. The only supported value is loadBalancer. |
platform.networkSecurityGroupId |
Yes | The Azure resource ID of the network security group. |
platform.operatorsAuthentication |
Yes | The managed identity assignments for the cluster operators. |
autoscaling.maxNodesTotal |
No | The maximum number of nodes the cluster autoscaler can create. Default: 0 (no limit). |
autoscaling.maxPodGracePeriodSeconds |
No | The maximum seconds for graceful pod termination before scale-down. |
autoscaling.maxNodeProvisionTimeSeconds |
No | The maximum seconds for node provisioning. Default: 900. |
autoscaling.podPriorityThreshold |
No | The priority threshold below which pods are treated as best-effort by the autoscaler. Default: -10. |
imageDigestMirrors |
No | Image digest mirror entries that redirect image pulls from a source registry to mirror registries. Maximum of 240 entries. |
nodeDrainTimeoutMinutes |
No | The default node drain grace period for all node pools, in minutes. Range: 0 to 10080. Default: 0 (no time limit). |
identity.type |
Yes | The identity type. Must be UserAssigned. |
identity.userAssignedIdentities |
Yes | A map of user-assigned managed identity resource IDs. |
Node pool resource properties
The node pool resource (Microsoft.RedHatOpenShift/hcpOpenShiftClusters/nodePools) defines a group of worker nodes that share the same configuration.
| Property | Required | Description |
|---|---|---|
version.id |
Yes | The OpenShift patch version for this node pool. For example, 4.20.8. The node pool version must be a patch version within the cluster's minor version. To list available versions, see List available OpenShift versions. |
version.channelGroup |
Yes | The OpenShift update channel. The only supported option is stable. |
platform.vmSize |
Yes | The Azure virtual machine size for the nodes. To list available VM sizes, see List available VM sizes. |
platform.subnetId |
Yes | The Azure resource ID of the subnet. If you don't specify this property, the node pool uses the cluster's worker subnet. |
platform.osDisk.diskSizeGiB |
No | The OS disk size in GiB. Minimum: 16. |
platform.osDisk.diskStorageAccountType |
No | The disk storage account type. Valid values are Premium_LRS, StandardSSD_LRS, or Standard_LRS. |
platform.osDisk.encryptionSetId |
No | The Azure resource ID of a DiskEncryptionSet to encrypt OS disks. |
platform.osDisk.diskType |
No | The OS disk type. Valid values are Managed or Ephemeral. |
platform.availabilityZone |
No | The Azure availability zone for the node pool. |
platform.enableEncryptionAtHost |
No | Enables host-level encryption for VMs. Default: false. |
replicas |
Conditional | The number of nodes. Required if autoScaling isn't configured. |
autoScaling.min |
Conditional | The minimum node count. Required if replicas isn't configured. |
autoScaling.max |
Conditional | The maximum node count. Required if replicas isn't configured. |
autoRepair |
No | Enables health checks for nodes. Default: true. |
labels |
No | Kubernetes labels applied to nodes, as an array of key-value pairs. |
taints |
No | Kubernetes taints applied to nodes. Each taint has a key, value, and effect (NoSchedule, PreferNoSchedule, or NoExecute). |
nodeDrainTimeoutMinutes |
No | The node drain grace period for this node pool, in minutes. Overrides the cluster default. Range: 0 to 10080. |
List available OpenShift versions
The cluster version.id uses a minor version (for example, 4.20), while the node pool version.id uses a patch version (for example, 4.20.8).
To list the available OpenShift versions in your region, run the following command:
az aro hcp version list --location "${LOCATION}" --output table
List available VM sizes
The availability of VM sizes depends on the Azure region of your cluster. To list VM sizes available in your region, run the following command:
az vm list-sizes --location "${LOCATION}" --output table
For the list of supported worker node VM sizes for Azure Red Hat OpenShift with hosted control planes, see Supported virtual machine sizes. Use the regional list to confirm that a supported size is available in your cluster's location.