Mulai Cepat - Membuat Azure Red Hat OpenShift default dengan kluster sarana kontrol yang dihosting (pratinjau)

Artikel ini menjelaskan cara cepat membuat kluster Azure Red Hat OpenShift dengan bidang kontrol yang dihosting (HCP) menggunakan Azure CLI atau templat Bicep dengan pengaturan default.

Nota

Panduan memulai cepat ini akan membuat kluster dengan pengaturan bawaan. Untuk menyesuaikan opsi seperti visibilitas API, jenis ingress, atau Key Vault privat, lihat Membuat kluster Azure Red Hat OpenShift dengan control plane terhosting.

File Bicep menentukan semua sumber daya Azure yang diperlukan untuk kluster, termasuk infrastruktur jaringan, identitas terkelola, penetapan peran, konfigurasi kluster, dan kumpulan simpul.

Templat menyebarkan kluster dengan default berikut:

• Visibilitas server API: Publik (dapat diakses internet)
• Jenis Ingress: Publik (rute aplikasi yang dapat diakses internet)
• Jaringan: OVN-Kubernetes dengan rentang CIDR bawaan
• Enkripsi etcd: Kunci yang dikelola pelanggan (Azure Key Vault)
• Konektivitas keluar: Load balancer
• FIPS: Dinonaktifkan
• Kumpulan simpul: Dua Standard_D8s_v3 simpul pekerja dengan disk OS 64 GiB Premium_LRS

Perintah CLI membuat semua sumber daya Azure yang diperlukan untuk kluster, termasuk infrastruktur jaringan, identitas terkelola, penetapan peran, konfigurasi kluster, dan kumpulan simpul.

Perintah menyebarkan kluster dengan default berikut:

• Visibilitas server API: Publik (dapat diakses internet)
• Jenis Ingress: Publik (rute aplikasi yang dapat diakses internet)
• Jaringan: OVN-Kubernetes dengan rentang CIDR bawaan
• Enkripsi etcd: Kunci yang dikelola pelanggan (Azure Key Vault)
• Konektivitas keluar: Load balancer
• FIPS: Dinonaktifkan
• Kumpulan simpul: Dua Standard_D8s_v3 simpul pekerja dengan disk OS 64 GiB Premium_LRS

Menyiapkan lingkungan Anda

Sebelum Anda membuat kluster, verifikasi bahwa lingkungan Azure Anda memenuhi persyaratan berikut.

Azure CLI

Pastikan Anda menggunakan Azure CLI versi 2.67.0 atau yang lebih tinggi. Gunakan az --version untuk memeriksa versi yang terinstal. Untuk menginstal atau meningkatkan, lihat Install Azure CLI.

Ekstensi ARO HCP CLI

1. Pasang ekstensi .

   az extension add --name aro
   

2. Verifikasi bahwa ekstensi telah diinstal:

   az aro hcp -h
   

Kuota sumber daya

Azure Red Hat OpenShift dengan sarana kontrol yang dihosting membutuhkan setidaknya 20 inti untuk membuat dan menjalankan kluster. Kuota sumber daya Azure default untuk langganan Azure baru tidak memenuhi persyaratan ini. Untuk meminta peningkatan batas sumber daya Anda, lihat Kuota standar: Menambah batas berdasarkan seri VM. Pastikan bahwa Anda memiliki kuota yang cukup untuk keluarga VM yang Anda pilih dari ukuran VM node pekerja yang didukung.

Untuk memeriksa kuota saat ini untuk keluarga mesin virtual default:

LOCATION=eastus
az vm list-usage -l $LOCATION \
  --query "[?contains(name.value, 'standardDSv5Family')]" -o table

Izin

Anda memerlukan izin Kontributor dan Administrator Akses Pengguna, atau izin Pemilik, pada grup sumber daya atau langganan tempat Anda membuat kluster. Izin ini diperlukan untuk membuat grup sumber daya, jaringan virtual, identitas terkelola, dan penetapan peran.

Untuk informasi selengkapnya, lihat Memverifikasi izin Anda.

Mendaftarkan penyedia sumber daya

Daftarkan penyedia sumber daya berikut di langganan Azure Anda:

• Microsoft.RedHatOpenShift
• Microsoft.Compute
• Microsoft.Storage
• Microsoft.Authorization

Untuk memeriksa apakah penyedia sumber daya terdaftar:

az provider list --query "[?namespace=='Microsoft.RedHatOpenShift'].registrationState" \
  --output table

Jika Anda menemukan penyedia sumber daya yang tidak terdaftar, daftarkan penyedia sumber daya tersebut.

az provider register --namespace Microsoft.RedHatOpenShift --wait
az provider register --namespace Microsoft.Compute --wait
az provider register --namespace Microsoft.Storage --wait
az provider register --namespace Microsoft.Authorization --wait

Untuk informasi selengkapnya tentang cara mendaftarkan penyedia sumber daya, lihat Mendaftarkan penyedia sumber daya.

Mengatur variabel lingkungan

Atur variabel lingkungan berikut untuk menentukan nama sumber daya yang Anda buat. Ganti nilai placeholder dengan nilai Anda sendiri.

# Azure resource location and subscription
LOCATION="<location>"
SUBSCRIPTION_ID="$(az account show --query id --output tsv)"
CUSTOMER_RG_NAME="<resource-group-name>"

# Network resources
CUSTOMER_NSG="<nsg-name>"
CUSTOMER_VNET_NAME="<vnet-name>"
CUSTOMER_VNET_SUBNET1="<worker-subnet-name>"
CUSTOMER_VNET_INTEGRATION_SUBNET_NAME="<vnet-integration-subnet-name>"

# Cluster and node pool
CLUSTER_NAME="<cluster-name>"
NP_NAME="<node-pool-name>"
CLUSTER_VERSION="<major.minor>"    # Example: 4.22
NP_VERSION="<major.minor.patch>"   # Example: 4.22.1

# Key Vault and KMS key
KEYVAULT_NAME="<key-vault-name>"
KMS_KEY_NAME="etcd-kms-key"

Buat grup sumber daya

Buat grup sumber daya untuk menyimpan sumber daya kluster, jaringan virtual, dan identitas terkelola.

az group create \
  --name "${CUSTOMER_RG_NAME}" \
  --subscription "${SUBSCRIPTION_ID}" \
  --location "${LOCATION}"

Buat file Bicep

Buat file bernama azuredeploy.bicep dengan konten berikut. Templat Bicep ini mendefinisikan semua sumber daya yang diperlukan untuk menerapkan kluster Azure Red Hat OpenShift dengan control plane yang dihosting menggunakan pengaturan default, termasuk grup keamanan jaringan, jaringan virtual, identitas terkelola, penetapan peran, kluster, dan pool node.

@description('Network Security Group Name')
param customerNsgName string

@description('Virtual Network Name')
param customerVnetName string

@description('Subnet Name')
param customerVnetSubnetName string

@description('Virtual Network Integration Subnet Name')
param customerVirtualNetworkIntegrationSubnetName string

@description('Name of the cluster')
param clusterName string

@description('The name of the node pool')
param nodePoolName string

@description('The OpenShift version for the cluster (X.Y format, e.g. 4.22)')
param clusterVersion string

@description('The OpenShift version for the node pool (X.Y.Z format, e.g. 4.22.1)')
param nodePoolVersion string


var randomSuffix = toLower(uniqueString(clusterName))
var etcdEncryptionKeyName = 'etcd-data-kms-encryption-key'
var randomKeyVaultSuffix = toLower(uniqueString(resourceGroup().id))
var customerKeyVaultName string = 'cust-kv-${randomKeyVaultSuffix}'
var addressPrefix = '10.0.0.0/16'
var subnetPrefix = '10.0.0.0/24'
var virtualNetworkIntegrationSubnetPrefix = '10.0.1.0/24'


// Network Security Group
resource customerNsg 'Microsoft.Network/networkSecurityGroups@2023-05-01' = {
  name: customerNsgName
  location: resourceGroup().location
}

// Virtual network with worker subnet and VNet integration subnet
resource customerVnet 'Microsoft.Network/virtualNetworks@2023-05-01' = {
  name: customerVnetName
  location: resourceGroup().location
  properties: {
    addressSpace: {
      addressPrefixes: [
        addressPrefix
      ]
    }
    subnets: [
      {
        name: customerVnetSubnetName
        properties: {
          addressPrefix: subnetPrefix
          networkSecurityGroup: {
            id: customerNsg.id
          }
        }
      }
      {
        name: customerVirtualNetworkIntegrationSubnetName
        properties: {
          addressPrefix: virtualNetworkIntegrationSubnetPrefix
          networkSecurityGroup: {
            id: customerNsg.id
          }
          delegations: [
            {
              name: 'aro-hcp-delegation'
              properties: {
                serviceName: 'Microsoft.RedHatOpenShift/hcpOpenShiftClusters'
              }
            }
          ]
        }
      }
    ]
  }
}

resource subnet 'Microsoft.Network/virtualNetworks/subnets@2022-07-01' existing = {
  name: customerVnetSubnetName
  parent: customerVnet
}

resource vnetIntegrationSubnet 'Microsoft.Network/virtualNetworks/subnets@2022-07-01' existing = {
  name: customerVirtualNetworkIntegrationSubnetName
  parent: customerVnet
}

// Key Vault for etcd encryption
resource customerKeyVault 'Microsoft.KeyVault/vaults@2024-12-01-preview' = {
  name: customerKeyVaultName
  location: resourceGroup().location
  properties: {
    sku: {
      family: 'A'
      name: 'standard'
    }
    tenantId: subscription().tenantId
    publicNetworkAccess: 'Enabled'
    enableRbacAuthorization: true
  }
}

resource etcdEncryptionKey 'Microsoft.KeyVault/vaults/keys@2024-12-01-preview' = {
  parent: customerKeyVault
  name: etcdEncryptionKeyName
  properties: {
    kty: 'RSA'
    keySize: 2048
  }
}

//
// Control plane identities
//

// Reader
var readerRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  'acdd72a7-3385-48ef-bd42-f606fba81ae7'
)

//
// Cluster API Azure managed identity
//

resource clusterApiAzureMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-cp-cluster-api-azure-${randomSuffix}'
  location: resourceGroup().location
}

// Azure Red Hat OpenShift with hosted control planes Cluster API Provider
var hcpClusterApiProviderRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  '88366f10-ed47-4cc0-9fab-c8a06148393e'
)

resource hcpClusterApiProviderRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, clusterApiAzureMi.id, hcpClusterApiProviderRoleId, subnet.id)
  scope: subnet
  properties: {
    principalId: clusterApiAzureMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: hcpClusterApiProviderRoleId
  }
}

resource hcpClusterApiProviderRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, clusterApiAzureMi.id, hcpClusterApiProviderRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: clusterApiAzureMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: hcpClusterApiProviderRoleId
  }
}

resource serviceManagedIdentityReaderOnClusterApiAzureMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, clusterApiAzureMi.id)
  scope: clusterApiAzureMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: readerRoleId
  }
}

//
// Control plane operator managed identity
//

resource controlPlaneMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-cp-control-plane-${randomSuffix}'
  location: resourceGroup().location
}

// Azure Red Hat OpenShift with hosted control planes Control Plane Operator
var hcpControlPlaneOperatorRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  'fc0c873f-45e9-4d0d-a7d1-585aab30c6ed'
)

resource hcpControlPlaneOperatorVnetRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, controlPlaneMi.id, hcpControlPlaneOperatorRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: controlPlaneMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: hcpControlPlaneOperatorRoleId
  }
}

resource hcpControlPlaneOperatorNsgRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, controlPlaneMi.id, hcpControlPlaneOperatorRoleId, customerNsg.id)
  scope: customerNsg
  properties: {
    principalId: controlPlaneMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: hcpControlPlaneOperatorRoleId
  }
}

resource serviceManagedIdentityReaderOnControlPlaneMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, controlPlaneMi.id)
  scope: controlPlaneMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: readerRoleId
  }
}

//
// Cloud controller manager managed identity
//

resource cloudControllerManagerMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-cp-cloud-controller-manager-${randomSuffix}'
  location: resourceGroup().location
}

// Azure Red Hat OpenShift Cloud Controller Manager
var cloudControllerManagerRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  'a1f96423-95ce-4224-ab27-4e3dc72facd4'
)

resource cloudControllerManagerRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, cloudControllerManagerMi.id, cloudControllerManagerRoleId, subnet.id)
  scope: subnet
  properties: {
    principalId: cloudControllerManagerMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: cloudControllerManagerRoleId
  }
}

resource cloudControllerManagerRoleNsgAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, cloudControllerManagerMi.id, cloudControllerManagerRoleId, customerNsg.id)
  scope: customerNsg
  properties: {
    principalId: cloudControllerManagerMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: cloudControllerManagerRoleId
  }
}

resource cloudControllerManagerRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, cloudControllerManagerMi.id, cloudControllerManagerRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: cloudControllerManagerMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: cloudControllerManagerRoleId
  }
}

resource serviceManagedIdentityReaderOnCloudControllerManagerMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, cloudControllerManagerMi.id)
  scope: cloudControllerManagerMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: readerRoleId
  }
}

//
// Ingress managed identity
//

resource ingressMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-cp-ingress-${randomSuffix}'
  location: resourceGroup().location
}

// Azure Red Hat OpenShift Cluster Ingress Operator
var ingressOperatorRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  '0336e1d3-7a87-462b-b6db-342b63f7802c'
)

// Azure Red Hat OpenShift Image Registry Operator
var imageRegistryOperatorRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  '8b32b316-c2f5-4ddf-b05b-83dacd2d08b5'
)

resource ingressOperatorRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, ingressMi.id, ingressOperatorRoleId, subnet.id)
  scope: subnet
  properties: {
    principalId: ingressMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: ingressOperatorRoleId
  }
}

resource ingressOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, ingressMi.id, ingressOperatorRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: ingressMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: ingressOperatorRoleId
  }
}

resource serviceManagedIdentityReaderOnIngressMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, ingressMi.id)
  scope: ingressMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: readerRoleId
  }
}

//
// Disk CSI driver managed identity
//

resource diskCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-cp-disk-csi-driver-${randomSuffix}'
  location: resourceGroup().location
}

resource serviceManagedIdentityReaderOnDiskCsiDriverMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, diskCsiDriverMi.id)
  scope: diskCsiDriverMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: readerRoleId
  }
}

//
// File CSI driver managed identity
//

resource fileCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-cp-file-csi-driver-${randomSuffix}'
  location: resourceGroup().location
}

// Azure Red Hat OpenShift File Storage Operator
var fileStorageOperatorRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  '0d7aedc0-15fd-4a67-a412-efad370c947e'
)

resource fileStorageOperatorRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, fileCsiDriverMi.id, fileStorageOperatorRoleId, subnet.id)
  scope: subnet
  properties: {
    principalId: fileCsiDriverMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: fileStorageOperatorRoleId
  }
}

resource fileStorageOperatorRoleNsgAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, fileCsiDriverMi.id, fileStorageOperatorRoleId, customerNsg.id)
  scope: customerNsg
  properties: {
    principalId: fileCsiDriverMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: fileStorageOperatorRoleId
  }
}

resource fileStorageOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, fileCsiDriverMi.id, fileStorageOperatorRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: fileCsiDriverMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: fileStorageOperatorRoleId
  }
}

resource serviceManagedIdentityReaderOnFileCsiDriverMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, fileCsiDriverMi.id)
  scope: fileCsiDriverMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: readerRoleId
  }
}

//
// Image registry managed identity
//

resource imageRegistryMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-cp-image-registry-${randomSuffix}'
  location: resourceGroup().location
}

resource imageRegistryOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, imageRegistryMi.id, imageRegistryOperatorRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: imageRegistryMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: imageRegistryOperatorRoleId
  }
}

resource serviceManagedIdentityReaderOnImageRegistryMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, imageRegistryMi.id)
  scope: imageRegistryMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: readerRoleId
  }
}

//
// Cloud network config managed identity
//

resource cloudNetworkConfigMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-cp-cloud-network-config-${randomSuffix}'
  location: resourceGroup().location
}

// Azure Red Hat OpenShift Network Operator
var networkOperatorRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  'be7a6435-15ae-4171-8f30-4a343eff9e8f'
)

resource networkOperatorRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, cloudNetworkConfigMi.id, networkOperatorRoleId, subnet.id)
  scope: subnet
  properties: {
    principalId: cloudNetworkConfigMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: networkOperatorRoleId
  }
}

resource networkOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, cloudNetworkConfigMi.id, networkOperatorRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: cloudNetworkConfigMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: networkOperatorRoleId
  }
}

resource serviceManagedIdentityReaderOnCloudNetworkMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, cloudNetworkConfigMi.id)
  scope: cloudNetworkConfigMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: readerRoleId
  }
}

//
// KMS managed identity
//

resource kmsMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-cp-kms-${randomSuffix}'
  location: resourceGroup().location
}

// Key Vault Crypto User
var keyVaultCryptoUserRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  '12338af0-0e69-4776-bea7-57ae8d297424'
)

resource keyVaultCryptoUserToKeyVaultRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, kmsMi.id, keyVaultCryptoUserRoleId, customerKeyVault.id)
  scope: customerKeyVault
  properties: {
    principalId: kmsMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: keyVaultCryptoUserRoleId
  }
}

resource serviceManagedIdentityReaderOnKmsMi 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, readerRoleId, kmsMi.id)
  scope: kmsMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: readerRoleId
  }
}

//
// Data plane identities
//

// Azure Red Hat OpenShift Federated Credential
var federatedCredentialsRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  'ef318e2a-8334-4a05-9e4a-295a196c6a6e'
)

resource dpDiskCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-dp-disk-csi-driver-${randomSuffix}'
  location: resourceGroup().location
}

resource dpDiskCsiDriverMiFederatedCredentialsRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, dpDiskCsiDriverMi.id, federatedCredentialsRoleId)
  scope: dpDiskCsiDriverMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: federatedCredentialsRoleId
  }
}

resource dpFileCsiDriverMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-dp-file-csi-driver-${randomSuffix}'
  location: resourceGroup().location
}

resource dpFileCsiDriverMiFederatedCredentialsRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, dpFileCsiDriverMi.id, federatedCredentialsRoleId)
  scope: dpFileCsiDriverMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: federatedCredentialsRoleId
  }
}

resource dpFileCsiDriverFileStorageOperatorRoleSubnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, dpFileCsiDriverMi.id, fileStorageOperatorRoleId, subnet.id)
  scope: subnet
  properties: {
    principalId: dpFileCsiDriverMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: fileStorageOperatorRoleId
  }
}

resource dpFileCsiDriverFileStorageOperatorRoleNsgAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, dpFileCsiDriverMi.id, fileStorageOperatorRoleId, customerNsg.id)
  scope: customerNsg
  properties: {
    principalId: dpFileCsiDriverMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: fileStorageOperatorRoleId
  }
}

resource dpFileCsiDriverFileStorageOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, dpFileCsiDriverMi.id, fileStorageOperatorRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: dpFileCsiDriverMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: fileStorageOperatorRoleId
  }
}

resource dpImageRegistryMi 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-dp-image-registry-${randomSuffix}'
  location: resourceGroup().location
}

resource dpImageRegistryMiFederatedCredentialsRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, dpImageRegistryMi.id, federatedCredentialsRoleId)
  scope: dpImageRegistryMi
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: federatedCredentialsRoleId
  }
}

resource dpImageRegistryOperatorRoleVnetAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, dpImageRegistryMi.id, imageRegistryOperatorRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: dpImageRegistryMi.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: imageRegistryOperatorRoleId
  }
}

//
// Service managed identity
//

resource serviceManagedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${clusterName}-service-managed-identity-${randomSuffix}'
  location: resourceGroup().location
}

// Azure Red Hat OpenShift with hosted control planes Service Managed Identity
var hcpServiceManagedIdentityRoleId = subscriptionResourceId(
  'Microsoft.Authorization/roleDefinitions',
  'c0ff367d-66d8-445e-917c-583feb0ef0d4'
)

resource serviceManagedIdentityRoleAssignmentVnet 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, hcpServiceManagedIdentityRoleId, customerVnet.id)
  scope: customerVnet
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: hcpServiceManagedIdentityRoleId
  }
}

resource serviceManagedIdentityRoleAssignmentNSG 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(resourceGroup().id, serviceManagedIdentity.id, hcpServiceManagedIdentityRoleId, customerNsg.id)
  scope: customerNsg
  properties: {
    principalId: serviceManagedIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: hcpServiceManagedIdentityRoleId
  }
}


// Cluster resource
resource hcp 'Microsoft.RedHatOpenShift/hcpOpenShiftClusters@2026-09-01-preview' = {
  name: clusterName
  location: resourceGroup().location
  properties: {
    version: {
      id: clusterVersion
    }
    etcd: {
      dataEncryption: {
        keyManagementMode: 'CustomerManaged'
        customerManaged: {
          encryptionType: 'KMS'
          kms: {
            activeKey: {
              name: etcdEncryptionKeyName
              version: last(split(etcdEncryptionKey.properties.keyUriWithVersion, '/'))
            }
            vaultName: customerKeyVaultName
            visibility: 'Public'
          }
        }
      }
    }
    platform: {
      subnetId: subnet.id
      vnetIntegrationSubnetId: vnetIntegrationSubnet.id
      networkSecurityGroupId: customerNsg.id
      operatorsAuthentication: {
        userAssignedIdentities: {
          controlPlaneOperators: {
            'cluster-api-azure': clusterApiAzureMi.id
            'control-plane': controlPlaneMi.id
            'cloud-controller-manager': cloudControllerManagerMi.id
            ingress: ingressMi.id
            'disk-csi-driver': diskCsiDriverMi.id
            'file-csi-driver': fileCsiDriverMi.id
            'image-registry': imageRegistryMi.id
            'cloud-network-config': cloudNetworkConfigMi.id
            kms: kmsMi.id
          }
          dataPlaneOperators: {
            'disk-csi-driver': dpDiskCsiDriverMi.id
            'file-csi-driver': dpFileCsiDriverMi.id
            'image-registry': dpImageRegistryMi.id
          }
          serviceManagedIdentity: serviceManagedIdentity.id
        }
      }
    }
  }
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${serviceManagedIdentity.id}': {}
      '${clusterApiAzureMi.id}': {}
      '${controlPlaneMi.id}': {}
      '${cloudControllerManagerMi.id}': {}
      '${ingressMi.id}': {}
      '${diskCsiDriverMi.id}': {}
      '${fileCsiDriverMi.id}': {}
      '${imageRegistryMi.id}': {}
      '${cloudNetworkConfigMi.id}': {}
      '${kmsMi.id}': {}
    }
  }
  dependsOn: [
    hcpClusterApiProviderRoleSubnetAssignment
    hcpClusterApiProviderRoleVnetAssignment
    hcpControlPlaneOperatorVnetRoleAssignment
    hcpControlPlaneOperatorNsgRoleAssignment
    cloudControllerManagerRoleSubnetAssignment
    cloudControllerManagerRoleNsgAssignment
    cloudControllerManagerRoleVnetAssignment
    ingressOperatorRoleSubnetAssignment
    ingressOperatorRoleVnetAssignment
    fileStorageOperatorRoleSubnetAssignment
    fileStorageOperatorRoleNsgAssignment
    fileStorageOperatorRoleVnetAssignment
    imageRegistryOperatorRoleVnetAssignment
    networkOperatorRoleSubnetAssignment
    networkOperatorRoleVnetAssignment
    keyVaultCryptoUserToKeyVaultRoleAssignment
    dpDiskCsiDriverMiFederatedCredentialsRoleAssignment
    dpFileCsiDriverMiFederatedCredentialsRoleAssignment
    dpImageRegistryMiFederatedCredentialsRoleAssignment
    serviceManagedIdentityRoleAssignmentVnet
    serviceManagedIdentityRoleAssignmentNSG
    dpFileCsiDriverFileStorageOperatorRoleSubnetAssignment
    dpFileCsiDriverFileStorageOperatorRoleNsgAssignment
    dpFileCsiDriverFileStorageOperatorRoleVnetAssignment
    dpImageRegistryOperatorRoleVnetAssignment
    serviceManagedIdentityReaderOnControlPlaneMi
    serviceManagedIdentityReaderOnCloudControllerManagerMi
    serviceManagedIdentityReaderOnIngressMi
    serviceManagedIdentityReaderOnDiskCsiDriverMi
    serviceManagedIdentityReaderOnFileCsiDriverMi
    serviceManagedIdentityReaderOnImageRegistryMi
    serviceManagedIdentityReaderOnCloudNetworkMi
    serviceManagedIdentityReaderOnClusterApiAzureMi
    serviceManagedIdentityReaderOnKmsMi
  ]
}

// Node pool resource
resource nodepool 'Microsoft.RedHatOpenShift/hcpOpenShiftClusters/nodePools@2026-09-01-preview' = {
  parent: hcp
  name: nodePoolName
  location: resourceGroup().location
  properties: {
    version: {
      id: nodePoolVersion
    }
    platform: {
      subnetId: hcp.properties.platform.subnetId
      vmSize: 'Standard_D8s_v3'
    }
    replicas: 2
  }
}

Menerapkan file Bicep

Sebarkan templat Bicep untuk membuat kluster dan semua sumber daya yang diperlukan.

az deployment group create \
  --name 'aro-hcp' \
  --subscription "${SUBSCRIPTION_ID}" \
  --resource-group "${CUSTOMER_RG_NAME}" \
  --template-file azuredeploy.bicep \
  --parameters \
    customerNsgName="${CUSTOMER_NSG}" \
    customerVnetName="${CUSTOMER_VNET_NAME}" \
    customerVirtualNetworkIntegrationSubnetName="${CUSTOMER_VNET_INTEGRATION_SUBNET_NAME}" \
    customerVnetSubnetName="${CUSTOMER_VNET_SUBNET1}" \
    clusterName="${CLUSTER_NAME}" \
    nodePoolName="${NP_NAME}" \
    clusterVersion="${CLUSTER_VERSION}" \
    nodePoolVersion="${NP_VERSION}"

Penerapan biasanya memerlukan waktu 15 hingga 20 menit.

Verifikasi proses penyebaran

Setelah penyebaran selesai, verifikasi bahwa kluster dan kumpulan simpul berhasil dibuat.

1. Verifikasi bahwa kluster berhasil dibuat.

   az aro hcp cluster show \
     --name "${CLUSTER_NAME}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --query "properties.provisioningState" \
     --output tsv
   

   Ketika kluster siap, outputnya adalah Succeeded.

2. Pastikan bahwa node pool berhasil dibuat.

   az aro hcp cluster nodepool show \
     --cluster-name "${CLUSTER_NAME}" \
     --name "${NP_NAME}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --query "properties.provisioningState" \
     --output tsv
   

   Ketika kumpulan simpul siap, outputnya adalah Succeeded.

Buat infrastruktur jaringan.

Buat grup keamanan jaringan, jaringan virtual dengan subnet pekerja, dan subnet integrasi VNet dengan delegasi.

1. Buat kelompok keamanan jaringan.

   az network nsg create \
     --name "${CUSTOMER_NSG}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --location "${LOCATION}"
   

2. Buat jaringan virtual dengan subnet pekerja.

   az network vnet create \
     --name "${CUSTOMER_VNET_NAME}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --location "${LOCATION}" \
     --address-prefixes 10.0.0.0/16 \
     --subnet-name "${CUSTOMER_VNET_SUBNET1}" \
     --subnet-prefixes 10.0.0.0/24 \
     --nsg "${CUSTOMER_NSG}"
   

3. Buat subnet integrasi VNet dengan delegasi yang diperlukan.

   az network vnet subnet create \
     --name "${CUSTOMER_VNET_INTEGRATION_SUBNET_NAME}" \
     --vnet-name "${CUSTOMER_VNET_NAME}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --address-prefixes 10.0.1.0/24 \
     --network-security-group "${CUSTOMER_NSG}" \
     --delegations Microsoft.RedHatOpenShift/hcpOpenShiftClusters
   

4. Dapatkan ID sumber daya untuk sumber daya jaringan.

   NSG_ID=$(az network nsg show \
     --name "${CUSTOMER_NSG}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --query id --output tsv)
   
   SUBNET_ID=$(az network vnet subnet show \
     --name "${CUSTOMER_VNET_SUBNET1}" \
     --vnet-name "${CUSTOMER_VNET_NAME}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --query id --output tsv)
   
   VNET_ID=$(az network vnet show \
     --name "${CUSTOMER_VNET_NAME}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --query id --output tsv)
   
   VNET_INTEGRATION_SUBNET_ID=$(az network vnet subnet show \
     --name "${CUSTOMER_VNET_INTEGRATION_SUBNET_NAME}" \
     --vnet-name "${CUSTOMER_VNET_NAME}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --query id --output tsv)
   

Membuat identitas terkelola

Buat identitas terkelola yang ditetapkan pengguna untuk operator kluster dan identitas terkelola layanan. Untuk deskripsi setiap identitas dan tujuannya, lihat Identitas terkelola dan penetapan peran yang diperlukan.

# Control plane operator identities
az identity create --name "${CLUSTER_NAME}-cp-cluster-api-azure" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-control-plane" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-cloud-controller-manager" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-ingress" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-image-registry" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-cloud-network-config" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-cp-kms" --resource-group "${CUSTOMER_RG_NAME}"

# Data plane operator identities
az identity create --name "${CLUSTER_NAME}-dp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-dp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}"
az identity create --name "${CLUSTER_NAME}-dp-image-registry" --resource-group "${CUSTOMER_RG_NAME}"

# Service managed identity
az identity create --name "${CLUSTER_NAME}-service-managed-identity" --resource-group "${CUSTOMER_RG_NAME}"

Dapatkan ID sumber daya dan ID utama untuk identitas terkelola.

# Control plane identity resource IDs and principal IDs
CLUSTER_API_AZURE_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-cluster-api-azure" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CLUSTER_API_AZURE_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-cluster-api-azure" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

CONTROL_PLANE_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-control-plane" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CONTROL_PLANE_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-control-plane" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

CLOUD_CONTROLLER_MANAGER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-controller-manager" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CLOUD_CONTROLLER_MANAGER_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-controller-manager" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

INGRESS_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-ingress" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
INGRESS_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-ingress" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

DISK_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)

FILE_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
FILE_CSI_DRIVER_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

IMAGE_REGISTRY_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
IMAGE_REGISTRY_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

CLOUD_NETWORK_CONFIG_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-network-config" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
CLOUD_NETWORK_CONFIG_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-cloud-network-config" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

KMS_MI_ID=$(az identity show --name "${CLUSTER_NAME}-cp-kms" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
KMS_MI_PID=$(az identity show --name "${CLUSTER_NAME}-cp-kms" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

# Data plane identity resource IDs and principal IDs
DP_DISK_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-dp-disk-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)

DP_FILE_CSI_DRIVER_MI_ID=$(az identity show --name "${CLUSTER_NAME}-dp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
DP_FILE_CSI_DRIVER_MI_PID=$(az identity show --name "${CLUSTER_NAME}-dp-file-csi-driver" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

DP_IMAGE_REGISTRY_MI_ID=$(az identity show --name "${CLUSTER_NAME}-dp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
DP_IMAGE_REGISTRY_MI_PID=$(az identity show --name "${CLUSTER_NAME}-dp-image-registry" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

# Service managed identity resource ID and principal ID
SERVICE_MI_ID=$(az identity show --name "${CLUSTER_NAME}-service-managed-identity" --resource-group "${CUSTOMER_RG_NAME}" --query id --output tsv)
SERVICE_MI_PID=$(az identity show --name "${CLUSTER_NAME}-service-managed-identity" --resource-group "${CUSTOMER_RG_NAME}" --query principalId --output tsv)

Buat penetapan peran

Tetapkan peran Azure yang diperlukan untuk setiap identitas terkelola. Untuk deskripsi terperinci tentang setiap identitas dan penetapan peran yang diperlukan, lihat Identitas terkelola dan penetapan peran yang diperlukan.

1. Atur GUID definisi peran.

   READER_ROLE="acdd72a7-3385-48ef-bd42-f606fba81ae7"
   HCP_CLUSTER_API_PROVIDER_ROLE="88366f10-ed47-4cc0-9fab-c8a06148393e"
   HCP_CONTROL_PLANE_OPERATOR_ROLE="fc0c873f-45e9-4d0d-a7d1-585aab30c6ed"
   CLOUD_CONTROLLER_MANAGER_ROLE="a1f96423-95ce-4224-ab27-4e3dc72facd4"
   INGRESS_OPERATOR_ROLE="0336e1d3-7a87-462b-b6db-342b63f7802c"
   FILE_STORAGE_OPERATOR_ROLE="0d7aedc0-15fd-4a67-a412-efad370c947e"
   NETWORK_OPERATOR_ROLE="be7a6435-15ae-4171-8f30-4a343eff9e8f"
   IMAGE_REGISTRY_OPERATOR_ROLE="8b32b316-c2f5-4ddf-b05b-83dacd2d08b5"
   FEDERATED_CREDENTIAL_ROLE="ef318e2a-8334-4a05-9e4a-295a196c6a6e"
   HCP_SERVICE_MI_ROLE="c0ff367d-66d8-445e-917c-583feb0ef0d4"
   

2. Menetapkan peran operator sarana kontrol.

   # Cluster API Azure → ARO HCP Cluster API Provider on worker subnet and VNet
   az role assignment create --assignee-object-id "${CLUSTER_API_AZURE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CLUSTER_API_PROVIDER_ROLE}" --scope "${SUBNET_ID}"
   az role assignment create --assignee-object-id "${CLUSTER_API_AZURE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CLUSTER_API_PROVIDER_ROLE}" --scope "${VNET_ID}"
   
   # Control Plane → ARO HCP Control Plane Operator on VNet and NSG
   az role assignment create --assignee-object-id "${CONTROL_PLANE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CONTROL_PLANE_OPERATOR_ROLE}" --scope "${VNET_ID}"
   az role assignment create --assignee-object-id "${CONTROL_PLANE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_CONTROL_PLANE_OPERATOR_ROLE}" --scope "${NSG_ID}"
   
   # Cloud Controller Manager → ARO Cloud Controller Manager on worker subnet, NSG, and VNet
   az role assignment create --assignee-object-id "${CLOUD_CONTROLLER_MANAGER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${CLOUD_CONTROLLER_MANAGER_ROLE}" --scope "${SUBNET_ID}"
   az role assignment create --assignee-object-id "${CLOUD_CONTROLLER_MANAGER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${CLOUD_CONTROLLER_MANAGER_ROLE}" --scope "${NSG_ID}"
   az role assignment create --assignee-object-id "${CLOUD_CONTROLLER_MANAGER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${CLOUD_CONTROLLER_MANAGER_ROLE}" --scope "${VNET_ID}"
   
   # Ingress → ARO Cluster Ingress Operator on worker subnet and VNet
   az role assignment create --assignee-object-id "${INGRESS_MI_PID}" --assignee-principal-type ServicePrincipal --role "${INGRESS_OPERATOR_ROLE}" --scope "${SUBNET_ID}"
   az role assignment create --assignee-object-id "${INGRESS_MI_PID}" --assignee-principal-type ServicePrincipal --role "${INGRESS_OPERATOR_ROLE}" --scope "${VNET_ID}"
   
   # File CSI Driver → ARO File Storage Operator on worker subnet, NSG, and VNet
   az role assignment create --assignee-object-id "${FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${SUBNET_ID}"
   az role assignment create --assignee-object-id "${FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${NSG_ID}"
   az role assignment create --assignee-object-id "${FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${VNET_ID}"
   
   # Image Registry → ARO Image Registry Operator on VNet
   az role assignment create --assignee-object-id "${IMAGE_REGISTRY_MI_PID}" --assignee-principal-type ServicePrincipal --role "${IMAGE_REGISTRY_OPERATOR_ROLE}" --scope "${VNET_ID}"
   
   # Cloud Network Config → ARO Network Operator on worker subnet and VNet
   az role assignment create --assignee-object-id "${CLOUD_NETWORK_CONFIG_MI_PID}" --assignee-principal-type ServicePrincipal --role "${NETWORK_OPERATOR_ROLE}" --scope "${SUBNET_ID}"
   az role assignment create --assignee-object-id "${CLOUD_NETWORK_CONFIG_MI_PID}" --assignee-principal-type ServicePrincipal --role "${NETWORK_OPERATOR_ROLE}" --scope "${VNET_ID}"
   

3. Menetapkan peran identitas terkelola layanan.

   # Service MI → ARO HCP Service Managed Identity on VNet and NSG
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_SERVICE_MI_ROLE}" --scope "${VNET_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${HCP_SERVICE_MI_ROLE}" --scope "${NSG_ID}"
   
   # Service MI → Reader on each control plane identity
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CLUSTER_API_AZURE_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CONTROL_PLANE_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CLOUD_CONTROLLER_MANAGER_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${INGRESS_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${DISK_CSI_DRIVER_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${FILE_CSI_DRIVER_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${IMAGE_REGISTRY_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${CLOUD_NETWORK_CONFIG_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${READER_ROLE}" --scope "${KMS_MI_ID}"
   

4. Tetapkan peran operator bidang data.

   # Service MI → ARO Federated Credential on each data plane identity
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FEDERATED_CREDENTIAL_ROLE}" --scope "${DP_DISK_CSI_DRIVER_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FEDERATED_CREDENTIAL_ROLE}" --scope "${DP_FILE_CSI_DRIVER_MI_ID}"
   az role assignment create --assignee-object-id "${SERVICE_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FEDERATED_CREDENTIAL_ROLE}" --scope "${DP_IMAGE_REGISTRY_MI_ID}"
   
   # Data Plane File CSI Driver → ARO File Storage Operator on worker subnet, NSG, and VNet
   az role assignment create --assignee-object-id "${DP_FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${SUBNET_ID}"
   az role assignment create --assignee-object-id "${DP_FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${NSG_ID}"
   az role assignment create --assignee-object-id "${DP_FILE_CSI_DRIVER_MI_PID}" --assignee-principal-type ServicePrincipal --role "${FILE_STORAGE_OPERATOR_ROLE}" --scope "${VNET_ID}"
   
   # Data Plane Image Registry → ARO Image Registry Operator on VNet
   az role assignment create --assignee-object-id "${DP_IMAGE_REGISTRY_MI_PID}" --assignee-principal-type ServicePrincipal --role "${IMAGE_REGISTRY_OPERATOR_ROLE}" --scope "${VNET_ID}"
   

Membuat kunci Key Vault dan KMS

Buat Azure Key Vault dengan otorisasi RBAC dan kunci enkripsi RSA 2048-bit untuk enkripsi etcd yang dikelola pelanggan.

1. Buat Key Vault.

   az keyvault create \
     --name "${KEYVAULT_NAME}" \
     --resource-group "${CUSTOMER_RG_NAME}" \
     --location "${LOCATION}" \
     --enable-rbac-authorization true
   

2. Tetapkan diri Anda sebagai Key Vault Crypto Officer. Peran ini memberi Anda izin untuk membuat kunci enkripsi KMS.

   az role assignment create \
     --role "Key Vault Crypto Officer" \
     --assignee "$(az account show --query user.name -o tsv)" \
     --scope "$(az keyvault show --name "${KEYVAULT_NAME}" --resource-group "${CUSTOMER_RG_NAME}" --query id -o tsv)"
   

3. Buat kunci enkripsi KMS.

   az keyvault key create \
     --vault-name "${KEYVAULT_NAME}" \
     --name "${KMS_KEY_NAME}" \
     --kty RSA \
     --size 2048
   

4. Dapatkan versi utama.

   KMS_KEY_VERSION=$(az keyvault key show \
     --vault-name "${KEYVAULT_NAME}" \
     --name "${KMS_KEY_NAME}" \
     --query "key.kid" --output tsv | rev | cut -d'/' -f1 | rev)
   

5. Tetapkan peran Pengguna Kripto Key Vault ke identitas terkelola KMS pada Key Vault.

   KEY_VAULT_CRYPTO_USER_ROLE="12338af0-0e69-4776-bea7-57ae8d297424"
   KEYVAULT_ID=$(az keyvault show --name "${KEYVAULT_NAME}" --query id --output tsv)
   
   az role assignment create \
     --assignee-object-id "${KMS_MI_PID}" \
     --assignee-principal-type ServicePrincipal \
     --role "${KEY_VAULT_CRYPTO_USER_ROLE}" \
     --scope "${KEYVAULT_ID}"
   

Membuat kluster

Buat kluster dengan pengaturan default.

az aro hcp cluster create \
  --name "${CLUSTER_NAME}" \
  --resource-group "${CUSTOMER_RG_NAME}" \
  --location "${LOCATION}" \
  --version "${CLUSTER_VERSION}" \
  --subnet-id "${SUBNET_ID}" \
  --vnet-integration-subnet-id "${VNET_INTEGRATION_SUBNET_ID}" \
  --nsg "${NSG_ID}" \
  --kms-vault-name "${KEYVAULT_NAME}" \
  --vault-visibility Public \
  --kms-active-key "{name:${KMS_KEY_NAME},version:${KMS_KEY_VERSION}}" \
  --assign-service-managed-identity "${SERVICE_MI_ID}" \
  --assign-control-plane-operator-identity cluster-api-azure "${CLUSTER_API_AZURE_MI_ID}" \
  --assign-control-plane-operator-identity control-plane "${CONTROL_PLANE_MI_ID}" \
  --assign-control-plane-operator-identity cloud-controller-manager "${CLOUD_CONTROLLER_MANAGER_MI_ID}" \
  --assign-control-plane-operator-identity ingress "${INGRESS_MI_ID}" \
  --assign-control-plane-operator-identity disk-csi-driver "${DISK_CSI_DRIVER_MI_ID}" \
  --assign-control-plane-operator-identity file-csi-driver "${FILE_CSI_DRIVER_MI_ID}" \
  --assign-control-plane-operator-identity image-registry "${IMAGE_REGISTRY_MI_ID}" \
  --assign-control-plane-operator-identity cloud-network-config "${CLOUD_NETWORK_CONFIG_MI_ID}" \
  --assign-control-plane-operator-identity kms "${KMS_MI_ID}" \
  --assign-data-plane-operator-identity disk-csi-driver "${DP_DISK_CSI_DRIVER_MI_ID}" \
  --assign-data-plane-operator-identity file-csi-driver "${DP_FILE_CSI_DRIVER_MI_ID}" \
  --assign-data-plane-operator-identity image-registry "${DP_IMAGE_REGISTRY_MI_ID}"

Penerapan biasanya memerlukan waktu 15 hingga 20 menit.

Verifikasi bahwa kluster berhasil dibuat.

az aro hcp cluster show \
  --name "${CLUSTER_NAME}" \
  --resource-group "${CUSTOMER_RG_NAME}" \
  --query "properties.provisioningState" \
  --output tsv

Ketika kluster siap, outputnya adalah Succeeded.

Membuat kumpulan simpul

Setelah kluster dibuat, buat kumpulan simpul dengan dua Standard_D8s_v3 simpul pekerja.

az aro hcp cluster nodepool create \
  --cluster-name "${CLUSTER_NAME}" \
  --name "${NP_NAME}" \
  --resource-group "${CUSTOMER_RG_NAME}" \
  --replicas 2 \
  --vm-size Standard_D8s_v3 \
  --version "${NP_VERSION}"

Pastikan bahwa node pool berhasil dibuat.

az aro hcp cluster nodepool show \
  --cluster-name "${CLUSTER_NAME}" \
  --name "${NP_NAME}" \
  --resource-group "${CUSTOMER_RG_NAME}" \
  --query "properties.provisioningState" \
  --output tsv

Ketika kumpulan simpul siap, outputnya adalah Succeeded.

Langkah berikutnya

Menyambungkan ke Azure Red Hat OpenShift dengan kluster sarana kontrol yang dihosting