Secure access to the API server using authorized IP address ranges in Azure Kubernetes Service (AKS)
This article shows you how to use API server authorized IP address ranges feature to limit which IP addresses and CIDRs can access control plane.
The Kubernetes API server is the core of the Kubernetes control plane and is the central way to interact with and manage your clusters. To improve the security of your clusters and minimize the risk of attacks, we recommend limiting the IP address ranges that can access the API server. To do this, you can use the API server authorized IP ranges feature.
Before you begin
- You need the Azure CLI version 2.0.76 or later installed and configured. Run
az --version
to find the version. If you need to install or upgrade, see Install Azure CLI. - To learn what IP addresses to include when integrating your AKS cluster with Azure DevOps, see the Azure DevOps Allowed IP addresses and domain URLs article.
Limitations
The API server authorized IP ranges feature has the following limitations:
- The API server authorized IP ranges feature was moved out of preview in October 2019. For clusters created after the feature was moved out of preview, this feature is only supported on the Standard SKU load balancer. Any existing clusters on the Basic SKU load balancer with the API server authorized IP ranges feature enabled will continue to work as is. However, these clusters cannot be migrated to a Standard SKU load balancer. Existing clusters will continue to work if the Kubernetes version and control plane are upgraded.
- The API server authorized IP ranges feature isn't supported on private clusters.
- When using this feature with clusters that use Node Public IP, the node pools using Node Public IP must use public IP prefixes. The public IP prefixes must be added as authorized ranges.
Overview of API server authorized IP ranges
The Kubernetes API server exposes underlying Kubernetes APIs and provides the interaction for management tools like kubectl
and the Kubernetes dashboard. AKS provides a single-tenant cluster control plane with a dedicated API server. The API server is assigned a public IP address by default. You can control access using Kubernetes role-based access control (Kubernetes RBAC) or Azure RBAC.
To secure access to the otherwise publicly accessible AKS control plane / API server, you can enable and use authorized IP ranges. These authorized IP ranges only allow defined IP address ranges to communicate with the API server. Any requests made to the API server from an IP address that isn't part of these authorized IP ranges is blocked.
Create an AKS cluster with API server authorized IP ranges enabled
Important
By default, your cluster uses the Standard SKU load balancer which you can use to configure the outbound gateway. When you enable API server authorized IP ranges during cluster creation, the public IP for your cluster is allowed by default in addition to the ranges you specify. If you specify "" or no value for --api-server-authorized-ip-ranges
, API server authorized IP ranges is disabled. Note that if you're using PowerShell, use --api-server-authorized-ip-ranges=""
(with equals signs) to avoid any parsing issues.
Note
You should add these ranges to an allow list:
- The cluster egress IP address (firewall, NAT gateway, or other address, depending on your outbound type).
- Any range that represents networks that you'll administer the cluster from.
The upper limit for the number of IP ranges you can specify is 200.
The rules can take up to two minutes to propagate. Please allow up to that time when testing the connection.
When creating a cluster with API server authorized IP ranges enabled, you use the --api-server-authorized-ip-ranges
parameter to provide a list of authorized public IP address ranges. When you specify a CIDR range, start with the first IP address in the range. For example, 137.117.106.90/29 is a valid range, but make sure you specify the first IP address in the range, such as 137.117.106.88/29.
Create an AKS cluster with API server authorized IP ranges enabled using the
az aks create
command with the--api-server-authorized-ip-ranges
parameter. The following example creates a cluster named myAKSCluster in the resource group named myResourceGroup with API server authorized IP ranges enabled. The IP address ranges allowed are 73.140.245.0/24:az aks create --resource-group myResourceGroup --name myAKSCluster --vm-set-type VirtualMachineScaleSets --load-balancer-sku standard --api-server-authorized-ip-ranges 73.140.245.0/24 --generate-ssh-keys
Specify outbound IPs for a Standard SKU load balancer
When creating a cluster with API server authorized IP ranges enabled, you can also specify the outbound IP addresses or prefixes for the cluster using the --load-balancer-outbound-ips
or --load-balancer-outbound-ip-prefixes
parameters. All IPs provided in the parameters are allowed along with the IPs in the --api-server-authorized-ip-ranges
parameter.
Create an AKS cluster with API server authorized IP ranges enabled and specify the outbound IP addresses for the Standard SKU load balancer using the
--load-balancer-outbound-ips
parameter. The following example creates a cluster named myAKSCluster in the resource group named myResourceGroup with API server authorized IP ranges enabled and the outbound IP addresses<public-ip-id-1>
and<public-ip-id-2>
:az aks create --resource-group myResourceGroup --name myAKSCluster --vm-set-type VirtualMachineScaleSets --load-balancer-sku standard --api-server-authorized-ip-ranges 73.140.245.0/24 --load-balancer-outbound-ips <public-ip-id-1>,<public-ip-id-2> --generate-ssh-keys
Allow only the outbound public IP of the Standard SKU load balancer
When you enable API server authorized IP ranges during cluster creation, the outbound public IP for the Standard SKU load balancer for your cluster is also allowed by default in addition to the ranges you specify. To allow only the outbound public IP of the Standard SKU load balancer, you use 0.0.0.0/32 when specifying the --api-server-authorized-ip-ranges
parameter.
Create an AKS cluster with API server authorized IP ranges enabled and allow only the outbound public IP of the Standard SKU load balancer using the
--api-server-authorized-ip-ranges
parameter. The following example creates a cluster named myAKSCluster in the resource group named myResourceGroup with API server authorized IP ranges enabled and allows only the outbound public IP of the Standard SKU load balancer:az aks create --resource-group myResourceGroup --name myAKSCluster --vm-set-type VirtualMachineScaleSets --load-balancer-sku standard --api-server-authorized-ip-ranges 0.0.0.0/32 --generate-ssh-keys
Update an existing cluster's API server authorized IP ranges
Update an existing cluster's API server authorized IP ranges using the
az aks update
command with the--api-server-authorized-ip-ranges
parameter. The following example updates API server authorized IP ranges on the cluster named myAKSCluster in the resource group named myResourceGroup. The IP address range to authorize is 73.140.245.0/24:az aks update --resource-group myResourceGroup --name myAKSCluster --api-server-authorized-ip-ranges 73.140.245.0/24
To allow multiple IP address ranges, you can list several IP addresses, separated by commas.
az aks update --resource-group myResourceGroup --name myAKSCluster --api-server-authorized-ip-ranges 73.140.245.0/24,193.168.1.0/24,194.168.1.0/24
You can also use 0.0.0.0/32 when specifying the
--api-server-authorized-ip-ranges
parameter to allow only the public IP of the Standard SKU load balancer.
Disable authorized IP ranges
Disable authorized IP ranges using the
az aks update
command and specify an empty range""
for the--api-server-authorized-ip-ranges
parameter.az aks update --resource-group myResourceGroup --name myAKSCluster --api-server-authorized-ip-ranges ""
Find existing authorized IP ranges
Find existing authorized IP ranges using the
az aks show
command with the--query
parameter set toapiServerAccessProfile.authorizedIpRanges
.az aks show --resource-group myResourceGroup --name myAKSCluster --query apiServerAccessProfile.authorizedIpRanges
How to find my IP to include in --api-server-authorized-ip-ranges
?
You must add your development machines, tooling, or automation IP addresses to the AKS cluster list of approved IP ranges to access the API server from there.
Another option is to configure a jumpbox with the necessary tooling inside a separate subnet in the firewall's virtual network. This assumes your environment has a firewall with the respective network, and you've added the firewall IPs to authorized ranges. Similarly, if you've forced tunneling from the AKS subnet to the firewall subnet, having the jumpbox in the cluster subnet is also okay.
Retrieve your IP address using the following command:
# Retrieve your IP address CURRENT_IP=$(dig +short "myip.opendns.com" "@resolver1.opendns.com")
Add your IP address to the approved list using Azure CLI or Azure PowerShell:
# Add to AKS approved list using Azure CLI az aks update --resource-group $RG --name $AKSNAME --api-server-authorized-ip-ranges $CURRENT_IP/24,73.140.245.0/24 # Add to AKS approved list using Azure PowerShell Set-AzAksCluster -ResourceGroupName $RG -Name $AKSNAME -ApiServerAccessAuthorizedIpRange '$CURRENT_IP/24,73.140.245.0/24'
Note
The above example adds another IP address to the approved ranges. Note that it still includes the IP address from Update a cluster's API server authorized IP ranges. If you don't include your existing IP address, this command will replace it with the new one instead of adding it to the authorized ranges. To disable authorized IP ranges, use az aks update
and specify an empty range "".
Another option is to use the following command on Windows systems to get the public IPv4 address, or you can follow the steps in Find your IP address.
Invoke-RestMethod http://ipinfo.io/json | Select -exp ip
You can also find this address by searching on what is my IP address in an internet browser.
Use Service Tags for API Server authorized IP ranges - (Preview)
Service tags are a convenient way to specify a group of IP addresses corresponding to a Kubernetes Service. You can use service tags to specify the IP addresses of Kubernetes services and specific IP addresses in the authorized IP ranges for the API server by separating them with a comma.
Limitations
- This feature is not compatible with API Server VNet Integration.
Important
AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:
Install the Azure CLI preview extension
Install the Azure CLI preview extension using the [az extension add][az-extension-add] command.
az extension add --name aks-preview
Update the extension to make sure you have the latest version using the [az extension update][az-extension-update] command.
az extension update --name aks-preview
Register the Service Tag Authorized IP feature flag
Register the EnableServiceTagAuthorizedIPPreview feature flag using the [az feature register][az-feature-register] command.
az feature register --namespace "Microsoft.ContainerService" --name "EnableServiceTagAuthorizedIPPreview"
It takes a few minutes for the registration to complete.
Verify the registration using the [az feature show][az-feature-show] command.
az feature show --namespace "Microsoft.ContainerService" --name "EnableServiceTagAuthorizedIPPreview"
Create an AKS cluster with Service Tag authorized IP ranges
Create a cluster with Service Tag authorized IP ranges using the --api-server-authorized-ip-ranges
parameter with the service tag AzureCloud
to allow all Azure services to access the API server and specify an additional IP address.
Note
Only one service tag is allowed in the --api-server-authorized-ip-ranges
parameter. You cannot specify multiple service tags.
az aks create --resource-group myResourceGroup \
--name myAKSCluster \
--api-server-authorized-ip-ranges AzureCloud,20.20.20.20
You should be able to curl the API server from an Azure VM or Azure service that is part of the AzureCloud
service tag.
Next steps
In this article, you enabled API server authorized IP ranges. This approach is one part of how you can securely run an AKS cluster. For more information, see Security concepts for applications and clusters in AKS and Best practices for cluster security and upgrades in AKS.
Azure Kubernetes Service