Edit

Web Application Routing (Preview)

  • Article
  • 11 minutes to read

The Web Application Routing add-on configures an Ingress controller in your Azure Kubernetes Service (AKS) cluster with SSL termination through certificates stored in Azure Key Vault. Optionally, it also integrates with Open Service Mesh (OSM) for end-to-end encryption of inter cluster communication using mutual TLS (mTLS). As applications are deployed, the add-on creates publicly accessible DNS names for endpoints.

Important

AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:

Limitations

  • Web Application Routing currently doesn't support named ports in ingress backend.

Web Application Routing add-on overview

The add-on deploys the following components:

  • nginx ingress controller: The ingress controller exposed to the internet.
  • external-dns controller: Watches for Kubernetes Ingress resources and creates DNS A records in the cluster-specific DNS zone. Note that this is only deployed when you pass in the --dns-zone-resource-id argument.

Prerequisites

Install the aks-preview Azure CLI extension

You also need the aks-preview Azure CLI extension version 0.5.75 or later. Install the aks-preview Azure CLI extension by using the az extension add command. Or install any available updates by using the az extension update command.

# Install the aks-preview extension
az extension add --name aks-preview

# Update the extension to make sure you have the latest version installed
az extension update --name aks-preview

Create and export a self-signed SSL certificate (if you don't already own one)

If you already have an SSL certificate, you can skip this step, otherwise you can use these commands to create a self-signed SSL certificate to use with the Ingress. You will need to replace <Hostname> with the DNS name that you will be using.

# Create a self-signed SSL certificate
openssl req -new -x509 -nodes -out aks-ingress-tls.crt -keyout aks-ingress-tls.key -subj "/CN=<Hostname>" -addext "subjectAltName=DNS:<Hostname>"

# Export the SSL certificate, skipping the password prompt
openssl pkcs12 -export -in aks-ingress-tls.crt -inkey aks-ingress-tls.key -out aks-ingress-tls.pfx

Create an Azure Key Vault to store the certificate

If you don't already have an Azure Key Vault, use this command to create one. Azure Key Vault is used to securely store the SSL certificates that will be loaded into the Ingress.

az keyvault create -g <ResourceGroupName> -l <Location> -n <KeyVaultName>

Import certificate to Azure Key Vault

Import the SSL certificate into Azure Key Vault.

az keyvault certificate import --vault-name <KeyVaultName> -n <KeyVaultCertificateName> -f aks-ingress-tls.pfx

Create an Azure DNS zone

If you want the add-on to automatically manage creating hostnames via Azure DNS, you need to create an Azure DNS zone if you don't have one already.

# Create a DNS zone
az network dns zone create -g <ResourceGroupName> -n <ZoneName>

Enable Web Application Routing via the Azure CLI

The Web Application Routing routing add-on can be enabled with the Azure CLI when deploying an AKS cluster. To do so, use the az aks create command with the --enable-addons argument. You can also enable Web Application Routing on an existing AKS cluster using the az aks enable-addons command.

The following additional add-ons are required:

  • azure-keyvault-secrets-provider: The Secret Store CSI provider for Azure Key Vault is required to retrieve the certificates from Azure Key Vault.
  • open-service-mesh: If you require encrypted intra cluster traffic (recommended) between the nginx ingress and your services, the Open Service Mesh add-on is required which provides mutual TLS (mTLS).

Important

To enable the add-on to reload certificates from Azure Key Vault when they change, you should to enable the secret autorotation feature of the Secret Store CSI driver with the --enable-secret-rotation argument. When the autorotation is enabled, the driver updates the pod mount and the Kubernetes secret by polling for changes periodically, based on the rotation poll interval you can define. The default rotation poll interval is 2 minutes.

az aks create -g <ResourceGroupName> -n <ClusterName> -l <Location> --enable-addons azure-keyvault-secrets-provider,open-service-mesh,web_application_routing --generate-ssh-keys --enable-secret-rotation

To enable Web Application Routing on an existing cluster, add the --addons parameter and specify web_application_routing as shown in the following example:

az aks enable-addons -g <ResourceGroupName> -n <ClusterName> --addons azure-keyvault-secrets-provider,open-service-mesh,web_application_routing --enable-secret-rotation

Note

To use the add-on with Open Service Mesh, you should install the osm command-line tool. This command-line tool contains everything needed to configure and manage Open Service Mesh. The latest binaries are available on the OSM GitHub releases page.

Retrieve the add-on's managed identity object ID

Retrieve user managed identity object ID for the add-on. This will be used in the next steps to grant permissions against the Azure DNS zone and the Azure Key Vault. Provide your <ResourceGroupName>, <ClusterName>, and <Location> in the script below which will retrieve the managed identity's object ID.

# Provide values for your environment
RGNAME=<ResourceGroupName>
CLUSTERNAME=<ClusterName>
LOCATION=<Location>

# Retrieve user managed identity object ID for the add-on
SUBSCRIPTION_ID=$(az account show --query id --output tsv)
MANAGEDIDENTITYNAME="webapprouting-${CLUSTERNAME}"
MCRGNAME=$(az aks show -g ${RGNAME} -n ${CLUSTERNAME} --query nodeResourceGroup -o tsv)
USERMANAGEDIDENTITY_RESOURCEID="/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${MCRGNAME}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/${MANAGEDIDENTITYNAME}"
MANAGEDIDENTITY_OBJECTID=$(az resource show --id $USERMANAGEDIDENTITY_RESOURCEID --query "properties.principalId" -o tsv | tr -d '[:space:]')

Configure the add-on to use Azure DNS to manage creating DNS zones

If you are going to use Azure DNS, update the add-on to pass in the --dns-zone-resource-id.

Retrieve the resource ID for the DNS zone.

ZONEID=$(az network dns zone show -g <ResourceGroupName> -n <ZoneName> --query "id" --output tsv)

Grant DNS Zone Contributor permissions on the DNS zone to the add-on's managed identity.

az role assignment create --role "DNS Zone Contributor" --assignee $MANAGEDIDENTITY_OBJECTID --scope $ZONEID

Update the add-on to enable the integration with Azure DNS. This will create the external-dns controller.

az aks addon update -g <ResourceGroupName> -n <ClusterName> --addon web_application_routing --dns-zone-resource-id=$ZONEID

Grant the add-on permissions to retrieve certificates from Azure Key Vault

The Web Application Routing add-on creates a user created managed identity in the cluster resource group. This managed identity will need to be granted permissions to retrieve SSL certificates from the Azure Key Vault.

Grant GET permissions for the Web Application Routing add-on to retrieve certificates from Azure Key Vault:

az keyvault set-policy --name <KeyVaultName> --object-id $MANAGEDIDENTITY_OBJECTID --secret-permissions get --certificate-permissions get

Connect to your AKS cluster

To connect to the Kubernetes cluster from your local computer, you use kubectl, the Kubernetes command-line client.

If you use the Azure Cloud Shell, kubectl is already installed. You can also install it locally using the az aks install-cli command:

az aks install-cli

To configure kubectl to connect to your Kubernetes cluster, use the az aks get-credentials command.

az aks get-credentials -g <ResourceGroupName> -n <ClusterName>

Deploy an application

Web Application Routing uses annotations on Kubernetes Ingress objects to create the appropriate resources, create records on Azure DNS (when configured), and retrieve the SSL certificates from Azure Key Vault.

Create the application namespace

For the sample application environment, let's first create a namespace called hello-web-app-routing to run the example pods:

kubectl create namespace hello-web-app-routing

We also need to add the application namespace to the OSM control plane:

osm namespace add hello-web-app-routing

Create the deployment

Create a file named deployment.yaml and copy in the following YAML.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: aks-helloworld  
spec:
  replicas: 1
  selector:
    matchLabels:
      app: aks-helloworld
  template:
    metadata:
      labels:
        app: aks-helloworld
    spec:
      containers:
      - name: aks-helloworld
        image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
        ports:
        - containerPort: 80
        env:
        - name: TITLE
          value: "Welcome to Azure Kubernetes Service (AKS)"

Create the service

Create a file named service.yaml and copy in the following YAML.

apiVersion: v1
kind: Service
metadata:
  name: aks-helloworld
spec:
  type: ClusterIP
  ports:
  - port: 80
  selector:
    app: aks-helloworld

Create the ingress

The Web Application Routing add-on creates an Ingress class on the cluster called webapprouting.kubernetes.azure.com . When you create an ingress object with this class, this will activate the add-on. To obtain the certificate URI to use in the Ingress from Azure Key Vault, run the following command.

az keyvault certificate show --vault-name <KeyVaultName> -n <KeyVaultCertificateName> --query "id" --output tsv

Create a file named ingress.yaml and copy in the following YAML.

Note

Update <Hostname> with your DNS host name and <KeyVaultCertificateUri> with the ID returned from Azure Key Vault. secretName is the name of the secret that going to be generated to store the certificate. This is the certificate that's going to be presented in the browser.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.azure.com/tls-cert-keyvault-uri: <KeyVaultCertificateUri>
    kubernetes.azure.com/use-osm-mtls: "true"
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/configuration-snippet: |2-

      proxy_ssl_name "default.hello-web-app-routing.cluster.local";
    nginx.ingress.kubernetes.io/proxy-ssl-secret: kube-system/osm-ingress-client-cert
    nginx.ingress.kubernetes.io/proxy-ssl-verify: "on"
  name: aks-helloworld
  namespace: hello-web-app-routing
spec:
  ingressClassName: webapprouting.kubernetes.azure.com
  rules:
  - host: <Hostname>
    http:
      paths:
      - backend:
          service:
            name: aks-helloworld
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - <Hostname>
    secretName: keyvault-aks-helloworld

Create the ingress backend

Open Service Mesh (OSM) leverages its IngressBackend API to configure a backend service to accept ingress traffic from trusted sources. To proxy connections to HTTPS backends, we will configure the Ingress and IngressBackend configurations to use https as the backend protocol, and have OSM issue a certificate that Nginx will use as the client certificate to proxy HTTPS connections to TLS backends. The client certificate and CA certificate will be stored in a Kubernetes secret that Nginx will use to authenticate service mesh backends. For more information, refer to Open Service Mesh: Ingress with Kubernetes Nginx Ingress Controller.

Create a file named ingressbackend.yaml and copy in the following YAML.

apiVersion: policy.openservicemesh.io/v1alpha1
kind: IngressBackend
metadata:
  name: aks-helloworld
  namespace: hello-web-app-routing
spec:
  backends:
  - name: aks-helloworld
    port:
      number: 80
      protocol: https
    tls:
      skipClientCertValidation: false
  sources:
  - kind: Service
    name: nginx
    namespace: app-routing-system
  - kind: AuthenticatedPrincipal
    name: ingress-nginx.ingress.cluster.local

Create the resources on the cluster

Use the kubectl apply command to create the resources.

kubectl apply -f deployment.yaml -n hello-web-app-routing
kubectl apply -f service.yaml -n hello-web-app-routing
kubectl apply -f ingress.yaml -n hello-web-app-routing
kubectl apply -f ingressbackend.yaml -n hello-web-app-routing

The following example output shows the created resources:

deployment.apps/aks-helloworld created
service/aks-helloworld created
ingress.networking.k8s.io/aks-helloworld created
ingressbackend.policy.openservicemesh.io/aks-helloworld created

Verify the managed ingress was created

kubectl get ingress -n hello-web-app-routing

NAME             CLASS                                HOSTS               ADDRESS       PORTS     AGE
aks-helloworld   webapprouting.kubernetes.azure.com   myapp.contoso.com   20.51.92.19   80, 443   4m

Accessing the endpoint over a DNS hostname

If you have not configured Azure DNS integration, you will need to configure your own DNS provider with an A record pointing to the ingress IP address and the host name you configured for the ingress, for example myapp.contoso.com.

Remove Web Application Routing

First, remove the associated namespace:

kubectl delete namespace hello-web-app-routing

The Web Application Routing add-on can be removed using the Azure CLI. To do so run the following command, substituting your AKS cluster and resource group name. Be careful if you already have some of the other add-ons (open-service-mesh or azure-keyvault-secrets-provider) enabled on your cluster so that you don't accidentally disable them.

az aks disable-addons --addons web_application_routing --name myAKSCluster --resource-group myResourceGroup

When the Web Application Routing add-on is disabled, some Kubernetes resources may remain in the cluster. These resources include configMaps and secrets, and are created in the app-routing-system namespace. To maintain a clean cluster, you may want to remove these resources.