Edit

Use Image Integrity to validate signed images before deploying them to your Azure Kubernetes Service (AKS) clusters (Preview)

  • Article

Azure Kubernetes Service (AKS) and its underlying container model provide increased scalability and manageability for cloud native applications. With AKS, you can launch flexible software applications according to the runtime needs of your system. However, this flexibility can introduce new challenges.

In these application environments, using signed container images helps verify that your deployments are built from a trusted entity and that images haven't been tampered with since their creation. Image Integrity is a service that allows you to add an Azure Policy built-in definition to verify that only signed images are deployed to your AKS clusters.

Note

Image Integrity is a feature based on Ratify. On an AKS cluster, the feature name and property name is ImageIntegrity, while the relevant Image Integrity pods' names contain Ratify.

Important

AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:

Prerequisites

  • An Azure subscription. If you don't have an Azure subscription, you can create a free account.

  • Azure CLI or Azure PowerShell.

  • aks-preview CLI extension version 0.5.96 or later.

  • Ensure that the Azure Policy add-on for AKS is enabled on your cluster. If you don't have this add-on installed, see Install Azure Policy add-on for AKS.

  • An AKS cluster enabled with OIDC Issuer. To create a new cluster or update an existing cluster, see Configure an AKS cluster with OIDC Issuer.

  • The EnableImageIntegrityPreview and AKS-AzurePolicyExternalData feature flags registered on your Azure subscription. Register the feature flags using the following commands:

    1. Register the EnableImageIntegrityPreview and AKS-AzurePolicyExternalData feature flags using the az feature register command.

      # Register the EnableImageIntegrityPreview feature flag
az feature register --namespace "Microsoft.ContainerService" --name "EnableImageIntegrityPreview"

# Register the AKS-AzurePolicyExternalData feature flag
az feature register --namespace "Microsoft.ContainerService" --name "AKS-AzurePolicyExternalData"

      It may take a few minutes for the status to show as Registered.

    2. Verify the registration status using the az feature show command.

      # Verify the EnableImageIntegrityPreview feature flag registration status
az feature show --namespace "Microsoft.ContainerService" --name "EnableImageIntegrityPreview"

# Verify the AKS-AzurePolicyExternalData feature flag registration status
az feature show --namespace "Microsoft.ContainerService" --name "AKS-AzurePolicyExternalData"

    3. Once the status shows Registered, refresh the registration of the Microsoft.ContainerService resource provider using the az provider register command.

      az provider register --namespace Microsoft.ContainerService

Considerations and limitations

  • Your AKS clusters must run Kubernetes version 1.26 or above.
  • You shouldn't use this feature for production Azure Container Registry (ACR) registries or workloads.
  • Image Integrity supports a maximum of 200 unique signatures concurrently cluster-wide.
  • Notation is the only supported verifier.
  • Audit is the only supported verification policy effect.

How Image Integrity works

Screenshot showing the basic architecture for Image Integrity.

Image Integrity uses Ratify, Azure Policy, and Gatekeeper to validate signed images before deploying them to your AKS clusters. Enabling Image Integrity on your cluster deploys a Ratify pod. This Ratify pod performs the following tasks:

  1. Reconciles certificates from Azure Key Vault per the configuration you set up through Ratify CRDs.
  2. Accesses images stored in ACR when validation requests come from Azure Policy. To enable this experience, Azure Policy extends Gatekeeper, an admission controller webhook for Open Policy Agent (OPA).
  3. Determines whether the target image is signed with a trusted cert and therefore considered as trusted.
  4. AzurePolicy and Gatekeeper consume the validation results as the compliance state to decide whether to allow the deployment request.

Enable Image Integrity on your AKS cluster

Note

Image signature verification is a governance-oriented scenario and leverages Azure Policy to verify image signatures on AKS clusters at-scale. We recommend using AKS's Image Integrity built-in Azure Policy initiative, which is available in Azure Policy's built-in definition library.

  • Create a policy assignment with the AKS policy initiative [Preview]: Use Image Integrity to ensure only trusted images are deployed using the az policy assignment create command.

    export SCOPE="/subscriptions/${SUBSCRIPTION}/resourceGroups/${RESOURCE_GROUP}"
export LOCATION=$(az group show -n ${RESOURCE_GROUP} --query location -o tsv)

az policy assignment create --name 'deploy-trustedimages' --policy-set-definition 'af28bf8b-c669-4dd3-9137-1e68fdc61bd6' --display-name 'Audit deployment with unsigned container images' --scope ${SCOPE} --mi-system-assigned --role Contributor --identity-scope ${SCOPE} --location ${LOCATION}

    The Ratify pod deploys after you enable the feature.

Note

The policy deploys the Image Integrity feature on your cluster when it detects any update operation on the cluster. If you want to enable the feature immediately, you need to create a policy remediation using the az policy remediation create command.

assignment_id=$(az policy assignment show -n 'deploy-trustedimages' --scope ${SCOPE} --query id -o tsv)
az policy remediation create  -a "$assignment_id" --definition-reference-id deployAKSImageIntegrity -n remediation -g ${RESOURCE_GROUP}

Set up verification configurations

For Image Integrity to properly verify the target signed image, you need to set up Ratify configurations through K8s CRDs using kubectl.

In this article, we use a self-signed CA cert from the official Ratify documentation to set up verification configurations. For more examples, see Ratify CRDs.

  1. Create a VerifyConfig file named verify-config.yaml and copy in the following YAML:

    apiVersion: config.ratify.deislabs.io/v1beta1
kind: CertificateStore
metadata:
  name: certstore-inline
spec:
  provider: inline
  parameters:
    value: |
      -----BEGIN CERTIFICATE-----
      MIIDQzCCAiugAwIBAgIUDxHQ9JxxmnrLWTA5rAtIZCzY8mMwDQYJKoZIhvcNAQEL
      BQAwKTEPMA0GA1UECgwGUmF0aWZ5MRYwFAYDVQQDDA1SYXRpZnkgU2FtcGxlMB4X
      DTIzMDYyOTA1MjgzMloXDTMzMDYyNjA1MjgzMlowKTEPMA0GA1UECgwGUmF0aWZ5
      MRYwFAYDVQQDDA1SYXRpZnkgU2FtcGxlMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A
      MIIBCgKCAQEAshmsL2VM9ojhgTVUUuEsZro9jfI27VKZJ4naWSHJihmOki7IoZS8
      3/3ATpkE1lGbduJ77M9UxQbEW1PnESB0bWtMQtjIbser3mFCn15yz4nBXiTIu/K4
      FYv6HVdc6/cds3jgfEFNw/8RVMBUGNUiSEWa1lV1zDM2v/8GekUr6SNvMyqtY8oo
      ItwxfUvlhgMNlLgd96mVnnPVLmPkCmXFN9iBMhSce6sn6P9oDIB+pr1ZpE4F5bwa
      gRBg2tWN3Tz9H/z2a51Xbn7hCT5OLBRlkorHJl2HKKRoXz1hBgR8xOL+zRySH9Qo
      3yx6WvluYDNfVbCREzKJf9fFiQeVe0EJOwIDAQABo2MwYTAdBgNVHQ4EFgQUKzci
      EKCDwPBn4I1YZ+sDdnxEir4wHwYDVR0jBBgwFoAUKzciEKCDwPBn4I1YZ+sDdnxE
      ir4wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAgQwDQYJKoZIhvcNAQEL
      BQADggEBAGh6duwc1MvV+PUYvIkDfgj158KtYX+bv4PmcV/aemQUoArqM1ECYFjt
      BlBVmTRJA0lijU5I0oZje80zW7P8M8pra0BM6x3cPnh/oZGrsuMizd4h5b5TnwuJ
      hRvKFFUVeHn9kORbyQwRQ5SpL8cRGyYp+T6ncEmo0jdIOM5dgfdhwHgb+i3TejcF
      90sUs65zovUjv1wa11SqOdu12cCj/MYp+H8j2lpaLL2t0cbFJlBY6DNJgxr5qync
      cz8gbXrZmNbzC7W5QK5J7fcx6tlffOpt5cm427f9NiK2tira50HU7gC3HJkbiSTp
      Xw10iXXMZzSbQ0/Hj2BF4B40WfAkgRg=
      -----END CERTIFICATE-----
---
apiVersion: config.ratify.deislabs.io/v1beta1
kind: Store
metadata:
  name: store-oras
spec:
  name: oras
# If you want to you use Workload Identity for Ratify to access Azure Container Registry,
# uncomment the following lines, and fill the proper ClientID:
# See more: https://ratify.dev/docs/reference/oras-auth-provider
# parameters:
#  authProvider:
#    name: azureWorkloadIdentity
#    clientID: XXX
---
apiVersion: config.ratify.deislabs.io/v1beta1
kind: Verifier
metadata:
  name: verifier-notary-inline
spec:
  name: notation
  artifactTypes: application/vnd.cncf.notary.signature
  parameters:
    verificationCertStores:  # certificates for validating signatures
      certs: # name of the trustStore
        - certstore-inline # name of the certificate store CRD to include in this trustStore
    trustPolicyDoc: # policy language that indicates which identities are trusted to produce artifacts
      version: "1.0"
      trustPolicies:
        - name: default
          registryScopes:
            - "*"
          signatureVerification:
            level: strict
          trustStores:
            - ca:certs
          trustedIdentities:
            - "*"

  2. Apply the VerifyConfig to your cluster using the kubectl apply command.

    kubectl apply -f verify-config.yaml

Deploy sample images to your AKS cluster

  • Deploy a signed image using the kubectl run demo command.

    kubectl run demo-signed --image=ghcr.io/deislabs/ratify/notary-image:signed

    The following example output shows that Image Integrity allows the deployment:

    ghcr.io/deislabs/ratify/notary-image:signed
pod/demo-signed created

If you want to use your own images, see the guidance for image signing.

Disable Image Integrity

  • Disable Image Integrity on your cluster using the az aks update command with the --disable-image-integrity flag.

    az aks update -g myResourceGroup -n MyManagedCluster --disable-image-integrity

Remove policy initiative

  • Remove the policy initiative using the az policy assignment delete command.

    az policy assignment delete --name 'deploy-trustedimages'

Next steps

In this article, you learned how to use Image Integrity to validate signed images before deploying them to your Azure Kubernetes Service (AKS) clusters. If you want to learn how to sign your own containers, see Build, sign, and verify container images using Notary and Azure Key Vault (Preview).