Edit

Use an internal load balancer with Azure Kubernetes Service (AKS)

  • Article
  • 7 minutes to read

You can create and use an internal load balancer to restrict access to your applications in Azure Kubernetes Service (AKS). An internal load balancer does not have a public IP and makes a Kubernetes service accessible only to applications that can reach the private IP. These applications can be within the same VNET or in another VNET through VNET peering. This article shows you how to create and use an internal load balancer with AKS.

Note

Azure Load Balancer is available in two SKUs: Basic and Standard. The Standard SKU is used by default when you create an AKS cluster. When you create a LoadBalancer service type, you'll get the same load balancer type as when you provisioned the cluster. For more information, see Azure Load Balancer SKU comparison.

Before you begin

This article assumes that you have an existing AKS cluster. If you need an AKS cluster, you can create one using Azure CLI, Azure PowerShell, or the Azure portal.

You also need the Azure CLI version 2.0.59 or later. Run az --version to find the version. If you need to install or upgrade, see Install Azure CLI.

If you want to use an existing subnet or resource group, the AKS cluster identity needs permission to manage network resources. For information, see Use kubenet networking with your own IP address ranges in AKS or Configure Azure CNI networking in AKS. If you're configuring your load balancer to use an IP address in a different subnet, ensure the AKS cluster identity also has read access to that subnet.

For more information on permissions, see Delegate AKS access to other Azure resources.

Create an internal load balancer

To create an internal load balancer, create a service manifest named internal-lb.yaml with the service type LoadBalancer and the azure-load-balancer-internal annotation as shown in the following example:

apiVersion: v1
kind: Service
metadata:
  name: internal-app
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: internal-app

Deploy the internal load balancer using kubectl apply and specify the name of your YAML manifest.

kubectl apply -f internal-lb.yaml

This command creates an Azure load balancer in the node resource group that's connected to the same virtual network as your AKS cluster.

When you view the service details, the IP address of the internal load balancer is shown in the EXTERNAL-IP column. In this context, External refers to the external interface of the load balancer. It doesn't mean that it receives a public, external IP address. This IP address is dynamically assigned from the same subnet as the AKS cluster.

It may take a minute or two for the IP address to change from <pending> to an actual internal IP address, as shown in the following example:

kubectl get service internal-app

NAME           TYPE           CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
internal-app   LoadBalancer   10.0.248.59   10.240.0.7    80:30555/TCP   2m

Specify an IP address

When you specify an IP address for the load balancer, the specified IP address must reside in the same subnet as the AKS cluster, but it can't already be assigned to a resource. For example, you shouldn't use an IP address in the range designated for the Kubernetes subnet within the AKS cluster.

You can use the az network vnet subnet list Azure CLI command or the Get-AzVirtualNetworkSubnetConfig PowerShell cmdlet to get the subnets in your virtual network.

For more information on subnets, see Add a node pool with a unique subnet.

If you want to use a specific IP address with the load balancer, there are two ways:

Important

Adding the LoadBalancerIP property to the load balancer YAML manifest is deprecating following upstream Kubernetes. While current usage remains the same and existing services are expected to work without modification, we highly recommend setting service annotations instead.

  • Set service annotations: Use service.beta.kubernetes.io/azure-load-balancer-ipv4 for an IPv4 address and service.beta.kubernetes.io/azure-load-balancer-ipv6 for an IPv6 address.

    apiVersion: v1
kind: Service
metadata:
  name: internal-app
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-ipv4: 10.240.0.25
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: internal-app

  • Add the LoadBalancerIP property to the load balancer YAML manifest: Add the Service.Spec.LoadBalancerIP property to the load balancer YAML manifest. This field is deprecating following upstream Kubernetes, and it can't support dual-stack. Current usage remains the same and existing services are expected to work without modification.

    apiVersion: v1
kind: Service
metadata:
  name: internal-app
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
  type: LoadBalancer
  loadBalancerIP: 10.240.0.25
  ports:
  - port: 80
  selector:
    app: internal-app

When you view the service details, the IP address in the EXTERNAL-IP column should reflect your specified IP address.

kubectl get service internal-app

NAME           TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
internal-app   LoadBalancer   10.0.184.168   10.240.0.25   80:30225/TCP   4m

For more information on configuring your load balancer in a different subnet, see Specify a different subnet

Before you begin

You must have the following resources:

To attach an Azure Private Link service to an internal load balancer, create a service manifest named internal-lb-pls.yaml with the service type LoadBalancer and the azure-load-balancer-internal and azure-pls-create annotation as shown in the following example. For more options, refer to the Azure Private Link Service Integration design document.

apiVersion: v1
kind: Service
metadata:
  name: internal-app
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    service.beta.kubernetes.io/azure-pls-create: "true"
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: internal-app

Deploy the internal load balancer using kubectl apply and specify the name of your YAML manifest.

kubectl apply -f internal-lb-pls.yaml

This command creates an Azure load balancer in the node resource group that's connected to the same virtual network as your AKS cluster.

When you view the service details, the IP address of the internal load balancer is shown in the EXTERNAL-IP column. In this context, External refers to the external interface of the load balancer. It doesn't mean that it receives a public, external IP address.

It may take a minute or two for the IP address to change from <pending> to an actual internal IP address, as shown in the following example:

kubectl get service internal-app

NAME           TYPE           CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
internal-app   LoadBalancer   10.125.17.53  10.125.0.66   80:30430/TCP   64m

A Private Link Service object is also created. This Private Link Service object connects to the frontend IP configuration of the load balancer associated with the Kubernetes service. You can get the details of the Private Link Service object with the following sample command:

# Create a variable for the resource group

AKS_MC_RG=$(az aks show -g myResourceGroup --name myAKSCluster --query nodeResourceGroup -o tsv)

# List the private link service

az network private-link-service list -g $AKS_MC_RG --query "[].{Name:name,Alias:alias}" -o table

Name      Alias
--------  -------------------------------------------------------------------------
pls-xyz   pls-xyz.abc123-defg-4hij-56kl-789mnop.eastus2.azure.privatelinkservice

A Private Endpoint allows you to privately connect to your Kubernetes service object via the Private Link Service you created. To do so, follow the sample commands.

# Create a variable for the private link service

AKS_PLS_ID=$(az network private-link-service list -g $AKS_MC_RG --query "[].id" -o tsv)

# Create the private endpoint

$ az network private-endpoint create \
    -g myOtherResourceGroup \
    --name myAKSServicePE \
    --vnet-name myOtherVNET \
    --subnet pe-subnet \
    --private-connection-resource-id $AKS_PLS_ID \
    --connection-name connectToMyK8sService

Use private networks

When you create your AKS cluster, you can specify advanced networking settings. These settings allow you to deploy the cluster into an existing Azure virtual network and subnets. For example, you can deploy your AKS cluster into a private network connected to your on-premises environment and run services that are only accessible internally.

For more information, see configure your own virtual network subnets with Kubenet or with Azure CNI.

You don't need to make any changes to the previous steps to deploy an internal load balancer that uses a private network in an AKS cluster. The load balancer is created in the same resource group as your AKS cluster, but it's instead connected to your private virtual network and subnet, as shown in the following example:

$ kubectl get service internal-app

NAME           TYPE           CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
internal-app   LoadBalancer   10.1.15.188   10.0.0.35     80:31669/TCP   1m

Note

You may need to assign a minimum of Microsoft.Network/virtualNetworks/subnets/read and Microsoft.Network/virtualNetworks/subnets/join/action permission to AKS MSI on the Azure Virtual Network resources. You can view the cluster identity with az aks show, such as az aks show --resource-group myResourceGroup --name myAKSCluster --query "identity". To create a role assignment, use the az role assignment create command.

Specify a different subnet

Add the azure-load-balancer-internal-subnet annotation to your service to specify a subnet for your load balancer. The subnet specified must be in the same virtual network as your AKS cluster. When deployed, the load balancer EXTERNAL-IP address is part of the specified subnet.

apiVersion: v1
kind: Service
metadata:
  name: internal-app
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    service.beta.kubernetes.io/azure-load-balancer-internal-subnet: "apps-subnet"
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: internal-app

Delete the load balancer

The load balancer will be deleted when all of its services are deleted.

As with any Kubernetes resource, you can directly delete a service, such as kubectl delete service internal-app, which also deletes the underlying Azure load balancer.

Next steps

To learn more about Kubernetes services, see the Kubernetes services documentation.