Create a Private AKS Cluster

Azure Public Test Date Azure Public Test Result

Azure US Gov Last Test Date Azure US Gov Last Test Result

Best Practice Check Cred Scan Check

Bicep Version

Deploy To Azure Deploy To Azure US Gov Visualize

This ARM template includes all the latest features like private AKS clusters, new simplified AKS-managed AAD integration, the brand new Azure RBAC for Kubernetes Authorization, actually in preview, and the use of managed identity in place of a service principal, and more.

The API server endpoint has no public IP address. To manage the API server, you will need to use a VM that has access to the AKS cluster's Azure Virtual Network (VNet). Therefore, the ARM template deploys a Jumpbox in the same virtual network that hosts the AKS private cluster. There are several options for establishing network connectivity to the private cluster.

  • Create a VM in the same Azure Virtual Network (VNet) as the AKS cluster.
  • Use a VM in a separate network and set up Virtual network peering. See the section below for more information on this option.
  • Use an Express Route or VPN connection.

Creating a VM in the same VNET as the AKS cluster is the easiest option. Express Route and VPNs add costs and require additional networking complexity. Virtual network peering requires you to plan your network CIDR ranges to ensure there are no overlapping ranges. For more information, see Create a private Azure Kubernetes Service cluster. For more information on Azure Private Links, see What is Azure Private Link?.

Architecture

The following picture shows the architecture and network topology of the sample.

Architecture

The ARM template deploys:

  • A new virtual network with three subnets, one for the AKS cluster, one for Azure Bastion and one for a Jumpbox virtual machine used to connect to the private AKS cluster
  • An AKS cluster with a private endpoint to the control plane / API server hosted by an AKS-managed Azure subscription. The cluster can communicate with the API server exposed via a Private Link Service using a private endpoint.
  • An Azure Bastion resource that provides secure and seamless SSH connectivity to the Jumpbox virtual machine directly in the Azure portal over SSL
  • A Private Endpoint in the same subnet of the AKS cluster.
  • A Network Interface associated to the private endpoint.
  • A Private DNS Zone for the name resolution of the private endpoint.
  • Two A records in the Private DNS Zone to let the cluster resolve the FQDN of the AKS cluster to the private IP address of its control plane.
  • A Virtual Network Link between the virtual network hosting the cluster and the Private DNS Zone to let the cluster to use the CNAME and A records defined by the Private DNS Zone for the name resolution of the API server of the cluster.
  • A Jumpbox virtual machine to manage the private AKS cluster
  • A Log Analytics workspace to collect the diagnostics logs and metrics of both the AKS cluster and Jumpbox virtual machine

Deployment

The following picture shows the resources deployed by the ARM template in the target resource group.

Resource Group

The following picture shows the resources deployed by the ARM template in the MC resource group associated to the AKS cluster:

MC Resource Group

Testing

If you open an ssh session to the Linux virtual machine and manually run the nslookup command using the FQND of the API server as a parameter, you should see an output like the the following:

Architecture

In order to connect the AKS cluster, you can run th following Bash script on the Jumpbox virtual machine:

#!/bin/bash
name="<name of the AKS cluster>"
resourceGroup="<name of the AKS resource group>"

# Install Azure CLI on Ubuntu
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

# Login with your Azure account
az login

# Install Kubectl
sudo az aks install-cli

# Use the following command to configure kubectl to connect to the new Kubernetes cluster
echo "Getting access credentials configure kubectl to connect to the ["$aksName"] AKS cluster..."
az aks get-credentials --name $name --resource-group $resourceGroup

Tags: Microsoft.Resources/deployments, Microsoft.Network/networkSecurityGroups, Microsoft.Network/virtualNetworks, Microsoft.Network/publicIPAddresses, Microsoft.Network/bastionHosts, Microsoft.OperationalInsights/workspaces, Microsoft.Storage/storageAccounts, Microsoft.Network/networkInterfaces, Microsoft.Compute/virtualMachines, Microsoft.Compute/virtualMachines/extensions, OmsAgentForLinux, DependencyAgentLinux, Microsoft.Network/privateDnsZones, Microsoft.Network/privateDnsZones/virtualNetworkLinks, Microsoft.Network/privateEndpoints, Microsoft.Network/privateEndpoints/privateDnsZoneGroups, Microsoft.ContainerService/managedClusters, SystemAssigned, [parameters('nodePoolType')]