Use Azure Key Vault secrets in Azure Pipelines

Azure DevOps Services | Azure DevOps Server 2022 - Azure DevOps Server 2019

Azure Key Vault allows developers to securely store and manage sensitive information like API keys, credentials, or certificates. Azure Key Vault service supports two types of containers: vaults and managed HSM (Hardware Security Module) pools. Vaults can store both software and HSM-backed keys, secrets, and certificates, while managed HSM pools exclusively support HSM-backed keys.

In this tutorial, you will learn how to:

• Create an Azure Key Vault using Azure CLI
• Add a secret and configure access to Azure key vault
• Use secrets in your pipeline

Prerequisites

• An Azure DevOps organization and a project. Create an organization or a project if you haven't already.

• An Azure subscription. Create an Azure account for free if you don't have one already.

Create a repo

If you already have your own repository, proceed to the next step. Otherwise, follow the instructions below to initialize your repository. We will use this Azure Repo to set up our pipeline.

1. Sign in to your Azure DevOps organization, and then navigate to your project.

2. Select Repos, and then select Initialize to initialize the main branch with a README.

   

Create an Azure Key Vault

1. Sign in to the Azure Portal, and then select the Cloud Shell button in the upper-right corner.

2. If you have more than one Azure subscription associated with your account, use the command below to specify a default subscription. You can use az account list to generate a list of your subscriptions.

   az account set --subscription <YOUR_SUBSCRIPTION_NAME_OR_ID>
   

3. Set your default Azure region. You can use az account list-locations to generate a list of available regions.

   az config set defaults.location=<YOUR_REGION>
   

4. Create a new resource group.

   az group create --name <YOUR_RESOURCE_GROUP_NAME>
   

5. Create a new Azure Key Vault.

   az keyvault create \
     --name <YOUR_KEY_VAULT_NAME> \
     --resource-group <YOUR_RESOURCE_GROUP_NAME>
   

6. Create a new secret in your Azure key vault.

   az keyvault secret set \
     --name <YOUR_SECRET_NAME> \
     --value <YOUR_ACTUAL_SECRET> \
     --vault-name <YOUR_KEY_VAULT_NAME>
   

Set up key vault access policies

To access our Azure Key Vault, we need to set up a service principal to grant access to Azure Pipelines. Follow this guide to create a service principal with Azure CLI, and then continue with the next steps in this section.

1. Navigate to Azure portal, and then use the search bar to find the key vault you created earlier.

   

2. Select Access policies, and then select Create to create a new policy.

3. Under Secret permissions, select Get and List.

4. Select Next, and then select the service principal you created earlier. A service principal is an object that represents an application or service that's requesting access to Azure resources.

5. Select Next, and then Next once more.

6. Review your policies, and then select Create when you're done.

Add role assignment

In the next step, we'll create an ARM service connection using service principal. Before we can verify the connection, we need to grant the service principal Read access at the subscription level:

1. Navigate to Azure portal

2. Select Subscriptions from the left navigation panel, and then find and select your subscription.

3. Select Access control, and then select Add > Add role assignment.

   

4. Select Reader under the Role tab, and then select Next.

5. Select User, group, or service principal, and then select Select members.

   

6. Use the search bar to find your service principal, and then select the "+" sign to select it, then click on the Select button.

7. Select Review + assign, review your settings, and then select Review + assign once more to confirm your choices and add the role assignment.

Create a service connection

1. Sign in to your Azure DevOps organization, and then navigate to your project.

2. Select Project settings > Service connections, and then select New service connection to create a new service connection.

3. Select Azure Resource Manager, and then select Next.

4. Select Service principal (manual), and then select Next.

5. Select Azure Cloud for Environment and Subscription for the Scope Level, then enter your Subscription Id and your Subscription Name.

6. Fill out the following fields with the information you obtained when creating the service principal, and then select Verify when you're done:

   • Service Principal Id: Your service principal appId.
   • Service Principal key: Your service principal password.
   • Tenant ID: Your service principal tenant.

7. Once the verification has succeeded, provide a name and description (optional) for your service connection, and then check the Grant access permission to all pipelines checkbox.

8. Select Verify and save when you're done.

   

Create a new pipeline

Classic

1. Sign in to your Azure DevOps organization, and then navigate to your project.

2. Select Pipelines, and then select New Pipeline.

3. Select Use the classic editor to create a classic pipeline.

4. Select Azure Repos Git, select your repository and default branch, and then select Continue.

5. Select the .Net Desktop pipeline template.

6. For this example, we will only need the last two tasks. Press CTRL and then select the first five tasks, right-click and choose Remove selected tasks(s) to delete them.

   

7. Select + to add a new task. Search for the Command line task, select it, and then select Add to add it to your pipeline. Once added, configure it as follows:

   • Display name: Create file
   • Script: echo $(YOUR_SECRET_NAME) > secret.txt

   

8. Select + to add a new task. Search for the Azure Key Vault task, select it, and then select Add* to add it to your pipeline. Once added, configure it as follows:

   • Display name: Azure Key Vault
   • Azure subscription: select your service principal service connection you created earlier
   • Key vault: select your key vault
   • Secret filter: A comma separated list of secret names or leave * to download all secrets from the selected key vault

   

9. Select the Copy files task and fill out the required fields as follows:

   • Display name: Copy File
   • Contents: secret.txt
   • Target Folder: $(build.artifactstagingdirectory)

   

10. Select the Publish Artifacts task and fill out the required fields as follows:

    • Display name: Publish Artifact
    • Path to publish: $(build.artifactstagingdirectory)
    • Artifact name: drop
    • Artifact publish location: Azure Pipelines

    

11. Select Save and queue, and then select Run to run your pipeline.

12. Once the pipeline run is complete, return to the pipeline summary and select the published artifact.

13. Select drop > secret.txt to download the published artifact.

    

14. Open the text file you just downloaded, the text file should contain the secret from your Azure key vault.

YAML

1. Sign in to your Azure DevOps organization, and then navigate to your project.

2. Select Pipelines, and then select New Pipeline.

3. Select Azure Repos Git (YAML), and then select your repository.

4. Select the Starter pipeline template.

5. The default pipeline will include a script that runs echo commands. Those are not needed so we can delete them.

6. Add the AzureKeyVault task, replacing the placeholders with the name of the service connection you created earlier and your key vault name. Your YAML file should resemble the following snippet:

   trigger:
   - main
   
   pool:
     vmImage: ubuntu-latest
   
   steps:
   - task: AzureKeyVault@2
     displayName: Azure Key Vault
     inputs:
       azureSubscription: 'YOUR_SERVICE_CONNECTION_NAME'
       KeyVaultName: 'YOUR_KEY_VAULT_NAME'
       SecretsFilter: '*'
       RunAsPreJob: false
   

7. Let's add the following tasks to copy and publish our secret. This example is for demonstration purposes only and should not be implemented in a production environment.

   trigger:
   - main
   
   pool:
     vmImage: ubuntu-latest
   
   steps:
   - task: AzureKeyVault@2
     displayName: Azure Key Vault
     inputs:
       azureSubscription: 'YOUR_SERVICE_CONNECTION_NAME'
       KeyVaultName: 'YOUR_KEY_VAULT_NAME'
       SecretsFilter: '*'
       RunAsPreJob: false
   
   - task: CmdLine@2
     displayName: Create file
     inputs:
       script: 'echo $(YOUR_SECRET_NAME) > secret.txt'
   
   - task: CopyFiles@2
     displayName: Copy file
     inputs:
       Contents: secret.txt
       targetFolder: '$(Build.ArtifactStagingDirectory)'
   
   - task: PublishBuildArtifacts@1
     displayName: Publish Artifact
     inputs:
       PathtoPublish: '$(Build.ArtifactStagingDirectory)'
       ArtifactName: 'drop'
       publishLocation: 'Container'
   

8. Select Save and run, and then select it once more to commit your changes and trigger the pipeline. You may be asked to allow the pipeline access to Azure resources, if prompted select Allow. You will only have to approve your pipeline once.

9. Select the CmdLine task to view the logs.

   

10. Once the pipeline run is complete, return to the pipeline summary and select the published artifact.

    

11. Select drop > secret.txt to download it.

    

12. Open the text file you just downloaded, the text file should contain the secret from your Azure key vault.

Warning

This tutorial is for educational purposes only. For security best practices and how to safely work with secrets, see Manage secrets in your server apps with Azure Key Vault.

Clean up resources

Follow the steps below to delete the resources you created:

1. If you've created a new organization to host your project, see how to delete your organization, otherwise delete your project.

2. All Azure resources created during this tutorial are hosted under a single resource group. Run the following command to delete your resource group and all of its resources.

   az group delete --name <YOUR_RESOURCE_GROUP_NAME>
   

FAQ

Q: I'm getting the following error: "the user or group does not have secrets list permission" what should I do?

A: If you encounter an error indicating that the user or group does not have secrets list permission on key vault, run the following commands to authorize your application to access the key or secret in the Azure Key Vault:

$ErrorActionPreference="Stop";
$Credential = Get-Credential;
Connect-AzAccount -SubscriptionId <YOUR_SUBSCRIPTION_ID> -Credential $Credential;
$spn=(Get-AzureRmADServicePrincipal -SPN <YOUR_SERVICE_PRINCIPAL_ID>);
$spnObjectId=$spn.Id;
Set-AzureRmKeyVaultAccessPolicy -VaultName key-vault-tutorial -ObjectId $spnObjectId -PermissionsToSecrets get,list;

Related articles

• Publish and download pipeline artifacts
• Release artifacts and artifact sources
• Use gates and approvals to control deployment