Build, sign, and verify container images using Notary and Azure Key Vault (Preview)

Article
12/15/2022
5 minutes to read

The Azure Key Vault (AKV) is used to store a signing key that can be utilized by notation with the notation AKV plugin (azure-kv) to sign and verify container images and other artifacts. The Azure Container Registry (ACR) allows you to attach these signatures using the az or oras CLI commands.

The signed containers enable users to assure deployments are built from a trusted entity and verify artifact hasn't been tampered with since their creation. The signed artifact ensures integrity and authenticity before the user pulls an artifact into any environment and avoid attacks.

In this tutorial:

Store a signing certificate in Azure Key Vault
Sign a container image with notation
Verify a container image signature with notation

Prerequisites

Create and sign in ACR with OCI artifact enabled

Create or use an Azure Key Vault

This tutorial can be run in the Azure Cloud Shell

Install the notation CLI and AKV plugin

Install notation v1.0.0-rc.1 with plugin support on a Linux environment. You can also download the package for other environments from the release page.

# Download, extract and install
curl -Lo notation.tar.gz https://github.com/notaryproject/notation/releases/download/v1.0.0-rc.1/notation_1.0.0-rc.1_linux_amd64.tar.gz
tar xvzf notation.tar.gz

# Copy the notation cli to the desired bin directory in your PATH
cp ./notation /usr/local/bin

Install the notation Azure Key Vault plugin for remote signing and verification.

Note

The plugin directory varies depending upon the operating system being used. The directory path below assumes Ubuntu. Please read the notation config article for more information.

# Create a directory for the plugin
mkdir -p ~/.config/notation/plugins/azure-kv

# Download the plugin
curl -Lo notation-azure-kv.tar.gz \
    https://github.com/Azure/notation-azure-kv/releases/download/v0.5.0-rc.1/notation-azure-kv_0.5.0-rc.1_Linux_amd64.tar.gz

# Extract to the plugin directory
tar xvzf notation-azure-kv.tar.gz -C ~/.config/notation/plugins/azure-kv notation-azure-kv

List the available plugins and verify that the plugin is available.
```
notation plugin ls
```

Configure environment variables

Note

For easy execution of commands in the tutorial, provide values for the Azure resources to match the existing ACR and AKV resources.

Configure AKV resource names.

# Name of the existing Azure Key Vault used to store the signing keys
AKV_NAME=<your-unique-keyvault-name>
# New desired key name used to sign and verify
KEY_NAME=wabbit-networks-io
CERT_SUBJECT="CN=wabbit-networks.io,O=Notary,L=Seattle,ST=WA,C=US"
CERT_PATH=./${KEY_NAME}.pem

Configure ACR and image resource names.

# Name of the existing registry example: myregistry.azurecr.io
ACR_NAME=myregistry
# Existing full domain of the ACR
REGISTRY=$ACR_NAME.azurecr.io
# Container name inside ACR where image will be stored
REPO=net-monitor
TAG=v1
IMAGE=$REGISTRY/${REPO}:$TAG
# Source code directory containing Dockerfile to build
IMAGE_SOURCE=https://github.com/wabbit-networks/net-monitor.git#main

Store the signing certificate in AKV

If you have an existing certificate, upload it to AKV. For more information on how to use your own signing key, see the signing certificate requirements. Otherwise create an x509 self-signed certificate storing it in AKV for remote signing using the steps below.

Create a self-signed certificate (Azure CLI)

Create a certificate policy file.

Once the certificate policy file is executed as below, it creates a valid signing certificate compatible with notation in AKV. The EKU listed is for code-signing, but isn't required for notation to sign artifacts. The subject is used later as trust identity that user tursts during verification.

cat <<EOF > ./my_policy.json
{
    "issuerParameters": {
    "certificateTransparency": null,
    "name": "Self"
    },
    "x509CertificateProperties": {
    "ekus": [
        "1.3.6.1.5.5.7.3.3"
    ],
    "keyUsage": [
        "digitalSignature"
    ],
    "subject": "$CERT_SUBJECT",
    "validityInMonths": 12
    }
}
EOF

Create the certificate.

az keyvault certificate create -n $KEY_NAME --vault-name $AKV_NAME -p @my_policy.json

Get the Key ID for the certificate.

KEY_ID=$(az keyvault certificate show -n $KEY_NAME --vault-name $AKV_NAME --query 'kid' -o tsv)

Download public certificate.

CERT_ID=$(az keyvault certificate show -n $KEY_NAME --vault-name $AKV_NAME --query 'id' -o tsv)
az keyvault certificate download --file $CERT_PATH --id $CERT_ID --encoding PEM

Add a signing key referencing the key id.

notation key add $KEY_NAME --plugin azure-kv --id $KEY_ID

List the keys to confirm.
```
notation key ls
```

Add the downloaded public certificate to named trust store for signature verification.

STORE_TYPE="ca"
STORE_NAME="wabbit-networks.io"
notation cert add --type $STORE_TYPE --store $STORE_NAME $CERT_PATH

List the certificate to confirm
```
notation cert ls
```

Build and sign a container image

Build and push a new image with ACR Tasks.

az acr build -r $ACR_NAME -t $IMAGE $IMAGE_SOURCE

Authenticate with your individual Azure AD identity to use an ACR token.

export USER_NAME="00000000-0000-0000-0000-000000000000"
export PASSWORD=$(az acr login --name $ACR_NAME --expose-token --output tsv --query accessToken)
notation login -u $USER_NAME -p $PASSWORD $REGISTRY

Sign the container image with the COSE signature format using the signing key added in previous step.
```
notation sign --signature-format cose --key $KEY_NAME $IMAGE
```
View the graph of signed images and associated signatures.
```
notation ls $IMAGE
```

View the graph of artifacts with the ORAS CLI (optional)

ACR support for OCI artifacts enables a linked graph of supply chain artifacts that can be viewed through the ORAS CLI or the Azure CLI.

Signed images can be view with the ORAS CLI.

oras login -u $USER_NAME -p $PASSWORD $REGISTRY
oras discover -o tree $IMAGE

View the graph of artifacts with the Azure CLI (optional)

List the manifest details for the container image.
```
az acr manifest show-metadata $IMAGE -o jsonc
```

Generates a result, showing the digest representing the notary v2 signature.

{
  "changeableAttributes": {
    "deleteEnabled": true,
    "listEnabled": true,
    "readEnabled": true,
    "writeEnabled": true
  },
  "createdTime": "2022-05-13T23:15:54.3478293Z",
  "digest": "sha256:effba96d9b7092a0de4fa6710f6e73bf8c838e4fbd536e95de94915777b18613",
  "lastUpdateTime": "2022-05-13T23:15:54.3478293Z",
  "name": "v1",
  "quarantineState": "Passed",
  "signed": false
}

Verify the container image

Configure trust policy before verification.

The trust policy is a JSON document named trustpolicy.json, which is stored under the notation configuration directory. Users who verify signed artifacts from a registry use the trust policy to specify trusted identities that sign the artifacts, and the level of signature verification to use.

Use the following command to configure trust policy for this tutorial. Upon successful execution of the command, one trust policy named wabbit-networks-images is created. This trust policy applies to all the artifacts stored in repositories defined in $REGISTRY/$REPO. The trust identity that user trusts has the x509 subject $CERT_SUBJECT from previous step, and stored under trust store named $STORE_NAME of type $STORE_TYPE. See Trust store and trust policy specification for details.
```
cat <<EOF > $HOME/.config/notation/trustpolicy.json
{
    "version": "1.0",
    "trustPolicies": [
        {
            "name": "wabbit-networks-images",
            "registryScopes": [ "$REGISTRY/$REPO" ],
            "signatureVerification": {
                "level" : "strict" 
            },
            "trustStores": [ "$STORE_TYPE:$STORE_NAME" ],
            "trustedIdentities": [
                "x509.subject: $CERT_SUBJECT"
            ]
        }
    ]
}
EOF
```
The notation command can also help to ensure the container image hasn't been tampered with since build time by comparing the sha with what is in the registry.
```
notation verify $IMAGE
```
Upon successful verification of the image using the trust policy, the sha256 digest of the verified image is returned in a successful output messages.

Next steps

See Enforce policy to only deploy signed container images to Azure Kubernetes Service (AKS) utilizing ratify and gatekeeper.