Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Note
This article describes how to manually configure TLS termination on a Gateway resource by authoring your own SecretProviderClass, mounting the Azure Key Vault provider for Secrets Store CSI Driver through a sample pod, and referencing the resulting Kubernetes Secret directly in the Gateway listener's certificateRefs. This approach is useful if you need full control over the certificate sync workflow or prefer to manage these resources yourself.
For most users, the recommended path is the automated Azure DNS and Azure Key Vault integration powered by the Application Routing operator, which provisions and reconciles the SecretProviderClass, the synced Kubernetes Secret, and the listener certificateRefs for you based on a pair of listener tls.options. To use the automated workflow, see Configure Azure DNS and TLS with the application routing Gateway API implementation.
The application routing add-on supports syncing secrets from Azure Key Vault (AKV) for securing Gateway API ingress traffic with TLS termination. Follow the steps below to create certificates and keys to terminate TLS traffic at the Gateway.
Prerequisites
- Enable the application routing Gateway API implementation
- Enable the Managed Gateway API Installation
- Set environment variables:
export CLUSTER=<cluster-name> export RESOURCE_GROUP=<resource-group-name> export LOCATION=<location>
Required client/server certificates and keys
- Create a root certificate and private key for signing the certificates for sample services:
mkdir httpbin_certs
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout httpbin_certs/example.com.key -out httpbin_certs/example.com.crt
- Generate a certificate and a private key for
httpbin.example.com:
openssl req -out httpbin_certs/httpbin.example.com.csr -newkey rsa:2048 -nodes -keyout httpbin_certs/httpbin.example.com.key -subj "/CN=httpbin.example.com/O=httpbin organization"
openssl x509 -req -sha256 -days 365 -CA httpbin_certs/example.com.crt -CAkey httpbin_certs/example.com.key -set_serial 0 -in httpbin_certs/httpbin.example.com.csr -out httpbin_certs/httpbin.example.com.crt
Configure a TLS ingress gateway
Set up Azure Key Vault and sync secrets to the cluster
Create Azure Key Vault
You need an Azure Key Vault resource to supply the certificate and key inputs to the application routing add-on.
export AKV_NAME=<azure-key-vault-resource-name> az keyvault create --name $AKV_NAME --resource-group $RESOURCE_GROUP --location $LOCATIONEnable Azure Key Vault provider for Secret Store CSI Driver add-on on your cluster.
az aks enable-addons --addons azure-keyvault-secrets-provider --resource-group $RESOURCE_GROUP --name $CLUSTERIf your Key Vault is using Azure RBAC for the permissions model, follow the instructions here to assign an Azure role of Key Vault Secrets User for the add-on's user-assigned managed identity. Alternatively, if your key vault is using the vault access policy permissions model, authorize the user-assigned managed identity of the add-on to access Azure Key Vault resource using access policy:
OBJECT_ID=$(az aks show --resource-group $RESOURCE_GROUP --name $CLUSTER --query 'addonProfiles.azureKeyvaultSecretsProvider.identity.objectId' -o tsv | tr -d '\r') CLIENT_ID=$(az aks show --resource-group $RESOURCE_GROUP --name $CLUSTER --query 'addonProfiles.azureKeyvaultSecretsProvider.identity.clientId') TENANT_ID=$(az keyvault show --resource-group $RESOURCE_GROUP --name $AKV_NAME --query 'properties.tenantId') az keyvault set-policy --name $AKV_NAME --object-id $OBJECT_ID --secret-permissions get listChoose how to supply the TLS certificate.
Azure Key Vault supports different object types. For this walkthrough, choose one of the following end-to-end options, and then continue with the common validation and Gateway steps.
Option Use this option when Key Vault stores Kubernetes Secret result Option 1: Store the certificate and key as Key Vault secrets You have separate PEM certificate ( .crt) and private key (.key) files, such as the files generated earlier in this article.Two Key Vault secrets. One synced Kubernetes TLS secret named httpbin-credential.Option 2: Reference a Key Vault certificate object directly Your certificate is already stored in Key Vault as a certificate object, such as an imported .pfx.One Key Vault certificate object. One synced Kubernetes TLS secret named httpbin-credential.Important
Don't follow both options. If you use Option 1, upload the certificate and key files as Key Vault secrets, and then apply the Option 1
SecretProviderClassmanifest. If you use Option 2, skip the Key Vault secret upload step and apply the Option 2SecretProviderClassmanifest.
Option 1: Store the certificate and key as Key Vault secrets
Use this option when you have separate PEM certificate (.crt) and private key (.key) files, such as the files generated earlier in this article.
Upload the certificate and key files as Key Vault secrets:
az keyvault secret set --vault-name $AKV_NAME --name test-httpbin-key --file httpbin_certs/httpbin.example.com.key az keyvault secret set --vault-name $AKV_NAME --name test-httpbin-crt --file httpbin_certs/httpbin.example.com.crtNote
Each time you rotate (change) the certificate or key, re-run this step to upload the new versions as secrets. To sync the updated values into the cluster automatically, enable autorotation on the Azure Key Vault provider for Secrets Store CSI Driver. For more information, see Autorotation and secret sync overview. If you prefer to have Key Vault manage the certificate lifecycle, use Option 2 instead.
Create the
SecretProviderClassthat references the two secrets you created in Key Vault:cat <<EOF | kubectl apply -f - apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: httpbin-credential-spc spec: provider: azure secretObjects: - secretName: httpbin-credential type: kubernetes.io/tls data: - objectName: test-httpbin-key key: tls.key - objectName: test-httpbin-crt key: tls.crt parameters: useVMManagedIdentity: "true" userAssignedIdentityID: $CLIENT_ID keyvaultName: $AKV_NAME cloudName: "" objects: | array: - | objectName: test-httpbin-key objectType: secret objectAlias: "test-httpbin-key" - | objectName: test-httpbin-crt objectType: secret objectAlias: "test-httpbin-crt" tenantId: $TENANT_ID EOF
Option 2: Reference a Key Vault certificate object directly
Use this option when the certificate is already stored in Azure Key Vault as a certificate object. You don't need to upload the .crt and .key files as separate Key Vault secrets.
In this example, test-httpbin-cert-pfx is the name of the certificate object in Azure Key Vault. To import an existing certificate into Key Vault as a certificate object, see Import a certificate into Key Vault. For more information about the object parameters, see obtain certificates and keys.
Create the
SecretProviderClassthat references the Key Vault certificate object:cat <<EOF | kubectl apply -f - apiVersion: secrets-store.csi.x-k8s.io/v1 kind: SecretProviderClass metadata: name: httpbin-credential-spc spec: provider: azure secretObjects: - secretName: httpbin-credential type: kubernetes.io/tls data: - objectName: test-httpbin-key key: tls.key - objectName: test-httpbin-crt key: tls.crt parameters: useVMManagedIdentity: "true" userAssignedIdentityID: $CLIENT_ID keyvaultName: $AKV_NAME cloudName: "" objects: | array: - | objectName: test-httpbin-cert-pfx #certificate object name from keyvault objectType: secret objectAlias: "test-httpbin-key" - | objectName: test-httpbin-cert-pfx #certificate object name from keyvault objectType: cert objectAlias: "test-httpbin-crt" tenantId: $TENANT_ID EOF
Sync the secret to the cluster
After you complete either Option 1 or Option 2, deploy a sample pod to sync the secret into the cluster. The Secrets Store CSI Driver requires a pod to reference the SecretProviderClass resource so that secrets sync from Azure Key Vault to the cluster.
Use the following manifest to deploy a sample pod:
cat <<EOF | kubectl apply -f - apiVersion: v1 kind: Pod metadata: name: secrets-store-sync-httpbin spec: containers: - name: busybox image: mcr.microsoft.com/oss/busybox/busybox:1.33.1 command: - "/bin/sleep" - "10" volumeMounts: - name: secrets-store01-inline mountPath: "/mnt/secrets-store" readOnly: true volumes: - name: secrets-store01-inline csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "httpbin-credential-spc" EOFVerify that the
httpbin-credentialsecret is created in thedefaultnamespace as defined in the SecretProviderClass resource.kubectl describe secret/httpbin-credentialExample output:
Name: httpbin-credential Namespace: default Labels: secrets-store.csi.k8s.io/managed=true Annotations: <none> Type: kubernetes.io/tls Data ==== tls.crt: 1180 bytes tls.key: 1675 bytes
Deploy TLS Gateway
Create a Kubernetes Gateway that references the
httpbin-credentialsecret under the TLS configuration:cat <<EOF | kubectl apply -f - apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: httpbin-gateway spec: gatewayClassName: approuting-istio listeners: - name: https hostname: "httpbin.example.com" port: 443 protocol: HTTPS tls: mode: Terminate certificateRefs: - name: httpbin-credential allowedRoutes: namespaces: from: Selector selector: matchLabels: kubernetes.io/metadata.name: default EOFThen, create a corresponding
HTTPRouteto configure the gateway's ingress traffic routes:cat <<EOF | kubectl apply -f - apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: httpbin spec: parentRefs: - name: httpbin-gateway hostnames: ["httpbin.example.com"] rules: - matches: - path: type: PathPrefix value: /status - path: type: PathPrefix value: /delay backendRefs: - name: httpbin port: 8000 EOFGet the gateway address and port:
kubectl wait --for=condition=programmed gateways.gateway.networking.k8s.io httpbin-gateway export INGRESS_HOST=$(kubectl get gateways.gateway.networking.k8s.io httpbin-gateway -o jsonpath='{.status.addresses[0].value}') export SECURE_INGRESS_PORT=$(kubectl get gateways.gateway.networking.k8s.io httpbin-gateway -o jsonpath='{.spec.listeners[?(@.name=="https")].port}')Send an HTTPS request to access the
httpbinservice:curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \ --cacert httpbin_certs/example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"You should see the httpbin service return the 418 I’m a Teapot code.