Continuous integration and continuous deployment to Azure IoT Edge devices (classic editor)

Applies to: IoT Edge 1.1 checkmark IoT Edge 1.1 IoT Edge 1.2 checkmark IoT Edge 1.2 IoT Edge 1.3 checkmark IoT Edge 1.3 IoT Edge 1.4 checkmark IoT Edge 1.4

Azure Pipelines includes a built-in Azure IoT Edge task that helps you adopt DevOps with your Azure IoT Edge applications. This article demonstrates how to use the continuous integration and continuous deployment features of Azure Pipelines to build, test, and deploy applications quickly and efficiently to your Azure IoT Edge using the classic editor. Alternatively, you can use YAML.

Diagram - CI and CD branches for development and production

In this article, you learn how to use the built-in Azure IoT Edge tasks for Azure Pipelines to create build and release pipelines for your IoT Edge solution. Each Azure IoT Edge task added to your pipeline implements one of the following four actions:

Action Description
Build module images Takes your IoT Edge solution code and builds the container images.
Push module images Pushes module images to the container registry you specified.
Generate deployment manifest Takes a deployment.template.json file and the variables, then generates the final IoT Edge deployment manifest file.
Deploy to IoT Edge devices Creates IoT Edge deployments to one or more IoT Edge devices.

Unless otherwise specified, the procedures in this article do not explore all the functionality available through task parameters. For more information, see the following resources:

Prerequisites

  • An Azure Repos repository. If you don't have one, you can Create a new Git repo in your project. For this article, we created a repository called IoTEdgeRepo.

  • An IoT Edge solution committed and pushed to your repository. If you want to create a new sample solution for testing this article, follow the steps in Develop and debug modules in Visual Studio Code or Develop and debug C# modules in Visual Studio. For this article, we created a solution in our repository called IoTEdgeSolution, which has the code for a module named filtermodule.

    For this article, all you need is the solution folder created by the IoT Edge templates in either Visual Studio Code or Visual Studio. You don't need to build, push, deploy, or debug this code before proceeding. You'll set up those processes in Azure Pipelines.

    Know the path to the deployment.template.json file in your solution, which is used in several steps. If you're unfamiliar with the role of the deployment template, see Learn how to deploy modules and establish routes.

    Tip

    If you're creating a new solution, clone your repository locally first. Then, when you create the solution you can choose to create it directly in the repository folder. You can easily commit and push the new files from there.

  • A container registry where you can push module images. You can use Azure Container Registry or a third-party registry.

  • An active Azure IoT hub with at least two IoT Edge devices for testing the separate test and production deployment stages. You can follow the quickstart articles to create an IoT Edge device on Linux or Windows

Create a build pipeline for continuous integration

In this section, you create a new build pipeline. You configure the pipeline to run automatically and publish build logs whenever you check in changes to the IoT Edge solution.

  1. Sign in to your Azure DevOps organization (https://dev.azure.com/{your organization}) and open the project that contains your IoT Edge solution repository.

    Open your DevOps project

  2. From the left pane menu in your project, select Pipelines. Select Create Pipeline at the center of the page. Or, if you already have build pipelines, select the New pipeline button in the top right.

    Create a new build pipeline

  3. At the bottom of the Where is your code? page, select Use the classic editor. If you wish to use YAML to create your project's build pipelines, see the YAML guide.

    Select Use the classic editor

  4. Follow the prompts to create your pipeline.

    1. Provide the source information for your new build pipeline. Select Azure Repos Git as the source, then select the project, repository, and branch where your IoT Edge solution code is located. Then, select Continue.

      Select your pipeline source

    2. Select Empty job instead of a template.

      Start with an empty job for your build pipeline

  5. Once your pipeline is created, you are taken to the pipeline editor. Here, you can change the pipeline's name, agent pool, and agent specification.

    You can select a Microsoft-hosted pool, or a self-hosted pool that you manage.

    In your pipeline description, choose the correct agent specification based on your target platform:

    • If you would like to build your modules in platform amd64 for Linux containers, choose ubuntu-18.04

    • If you would like to build your modules in platform amd64 for Windows 1809 containers, you need to set up self-hosted agent on Windows.

    • If you would like to build your modules in platform arm32v7 or arm64 for Linux containers, you need to set up self-hosted agent on Linux.

    Configure build agent specification.

  6. Your pipeline comes preconfigured with a job called Agent job 1. Select the plus sign (+) to add four tasks to the job: Azure IoT Edge twice, Copy Files once, and Publish Build Artifacts once. Search for each task and hover over the task's name to see the Add button.

    Add Azure IoT Edge task.

    When all four tasks are added, your Agent job looks like the following example:

    Four tasks in the build pipeline.

  7. Select the first Azure IoT Edge task to edit it. This task builds all modules in the solution with the target platform that you specify. Edit the task with the following values:

    Parameter Description
    Display name The display name is automatically updated when the Action field changes.
    Action Select Build module images.
    .template.json file Select the ellipsis (...) and navigate to the deployment.template.json file in the repository that contains your IoT Edge solution.
    Default platform Select the appropriate operating system for your modules based on your targeted IoT Edge device.
    Output variables Provide a reference name to associate with the file path where your deployment.json file generates, such as edge.

    For more information about this task and its parameters, see Azure IoT Edge task.

    These configurations use the image repository and tag that are defined in the module.json file to name and tag the module image. Build module images also helps replace the variables with the exact value you define in the module.json file. In Visual Studio or Visual Studio Code, you specify the actual value in a .env file. In Azure Pipelines, you set the value on the Pipeline Variables tab. Select the Variables tab on the pipeline editor menu and configure the name and value as following:

    • ACR_ADDRESS: Your Azure Container Registry Login server value. You can find the login server value on the container registry's overview page in the Azure portal.

    If you have other variables in your project, you can specify the name and value on this tab. Build module images recognizes only variables that are in ${VARIABLE} format. Make sure you use this format in your **/module.json files.

  8. Select the second Azure IoT Edge task to edit it. This task pushes all module images to the container registry that you select.

    Parameter Description
    Display name The display name is automatically updated when the Action field changes.
    Action Select Push module images.
    Container registry type Use the default type: Azure Container Registry.
    Azure subscription Choose your subscription.
    Azure Container Registry Select the type of container registry that you use to store your module images. Depending on which registry type you choose, the form changes. If you choose Azure Container Registry, use the dropdown lists to select the Azure subscription and the name of your container registry. If you choose Generic Container Registry, select New to create a registry service connection.
    .template.json file Select the ellipsis (...) and navigate to the deployment.template.json file in the repository that contains your IoT Edge solution.
    Default platform Select the appropriate operating system for your modules based on your targeted IoT Edge device.
    Add registry credential to deployment manifest Specify true to add the registry credential for pushing docker images to deployment manifest.

    For more information about this task and its parameters, see Azure IoT Edge task.

    If you have multiple container registries to host your module images, you need to duplicate this task, select different container registry, and use Bypass module(s) in the Advanced settings to bypass the images that are not for this specific registry.

  9. Select the Copy Files task to edit it. Use this task to copy files to the artifact staging directory.

    Parameter Description
    Display name Use the default name or customize
    Source folder The folder with the files to be copied.
    Contents Add two lines: deployment.template.json and **/module.json. These two files serve as inputs to generate the IoT Edge deployment manifest.
    Target Folder Specify the variable $(Build.ArtifactStagingDirectory). See Build variables to learn about the description.

    For more information about this task and its parameters, see Copy files task.

  10. Select the Publish Build Artifacts task to edit it. Provide artifact staging directory path to the task so that the path can be published to release pipeline.

    Parameter Description
    Display name Use the default name or customize.
    Path to publish Specify the variable $(Build.ArtifactStagingDirectory). See Build variables to learn more.
    Artifact name Use the default name: drop
    Artifact publish location Use the default location: Azure Pipelines

    For more information about this task and its parameters, see Publish build artifacts task.

  11. Open the Triggers tab and check the box to Enable continuous integration. Make sure the branch containing your code is included.

    Turn on continuous integration trigger

  12. Select Save from the Save & queue dropdown.

This pipeline is now configured to run automatically when you push new code to your repo. The last task, publishing the pipeline artifacts, triggers a release pipeline. Continue to the next section to build the release pipeline.

Create a release pipeline for continuous deployment

In this section, you create a release pipeline that is configured to run automatically when your build pipeline drops artifacts, and it will show deployment logs in Azure Pipelines.

Create a new pipeline, and add a new stage:

  1. In the Releases tab under Pipelines, choose + New pipeline. Or, if you already have release pipelines, choose the + New button and select + New release pipeline.

    Add a release pipeline using the + New pipeline button

  2. When prompted to select a template, choose to start with an Empty job.

    Start with an empty job for your release pipeline

  3. Your new release pipeline initializes with one stage, called Stage 1. Rename Stage 1 to dev and treat it as a continuous deployment pipeline for your development environment. Usually, continuous deployment pipelines have multiple stages including dev, staging, and prod. You can use different names and create more based on your DevOps practice. Close the stage details window once it's renamed.

    You can also rename your release pipeline by selecting the "New release pipeline" text at the top.

  4. Link the release to the build artifacts that are published by the build pipeline. Click Add in artifacts area.

    Click add in the artifacts area of the interface

  5. On the Add an artifact page, select Build as the Source type. Choose the project and the build pipeline you created. If you wish, you can change the Source alias to something more descriptive. Then, select Add.

    On the add an artifact page, select Add to create the artifact

  6. Open the artifact triggers and select the toggle to enable the continuous deployment trigger. Now, a new release will be created each time a new build is available.

    Open the artifact triggers and toggle to enable the continuous deployment trigger

  7. The dev stage is preconfigured with one job and zero tasks. From the pipeline menu, select Tasks then choose the dev stage. Select the Agent job and change its Display name to QA. You can configure details about the agent job, but the deployment task is platform insensitive so you can use any Agent specification in the chosen Agent pool.

    View the tasks for your dev stage under the Tasks tab

  8. On the QA job, select the plus sign (+) to add two tasks. Search for and add Azure IoT Edge twice.

  9. Select the first Azure IoT Edge task and configure it with the following values:

    Parameter Description
    Display name The display name is automatically updated when the Action field changes.
    Action Select Generate deployment manifest.
    .template.json file Specify the path: $(System.DefaultWorkingDirectory)/Drop/drop/deployment.template.json. The path is published from build pipeline.
    Default platform Select the appropriate operating system for your modules based on your targeted IoT Edge device.
    Output path Put the path $(System.DefaultWorkingDirectory)/Drop/drop/configs/deployment.json. This path is the final IoT Edge deployment manifest file.

    These configurations helps replace the module image URLs in the deployment.template.json file. The Generate deployment manifest also helps replace the variables with the exact value you defined in the deployment.template.json file. In VS/VS Code, you are specifying the actual value in a .env file. In Azure Pipelines, you set the value in Release Pipeline Variables tab. Move to Variables tab and configure the name and value as following:

    • ACR_ADDRESS: Your Azure Container Registry Login server value. You can retrieve the Login server from the Overview page of your container registry in the Azure portal.
    • ACR_PASSWORD: Your Azure Container Registry password.
    • ACR_USER: Your Azure Container Registry username.

    If you have other variables in your project, you can specify the name and value in this tab. The Generate deployment manifest can only recognize the variables that are in ${VARIABLE} flavor. Make sure you are using this flavor in your *.template.json files.

    "registryCredentials": {
      "<ACR name>": { // Your Azure Container Registry **Registry name** value
        "username": "${ACR_USER}",
        "password": "${ACR_PASSWORD}",
        "address": "${ACR_ADDRESS}"
      }
    }
    

    Configure the variables for your release pipeline in the Variables tab

  10. Select the second Azure IoT Edge task and configure it with the following values:

    Parameter Description
    Display name The display name is automatically updated when the Action field changes.
    Action Select Deploy to IoT Edge devices.
    Deployment file Put the path $(System.DefaultWorkingDirectory)/Drop/drop/configs/deployment.json. This path is the file IoT Edge deployment manifest file.
    Azure subscription Select the subscription that contains your IoT Hub.
    IoT Hub name Select your IoT hub.
    Choose single/multiple device Choose whether you want the release pipeline to deploy to one or multiple devices. If you deploy to a single device, enter the IoT Edge device ID. If you are deploying to multiple devices, specify the device target condition. The target condition is a filter to match a set of IoT Edge devices in IoT Hub. If you want to use device tags as the condition, you need to update your corresponding devices tags with IoT Hub device twin. Update the IoT Edge deployment ID and IoT Edge deployment priority in the advanced settings. For more information about creating a deployment for multiple devices, see Understand IoT Edge automatic deployments.
    Device ID or target condition Depending on the prior selection, specify a device ID or target condition to deploy to multiple devices.
    Advanced For the IoT Edge deployment ID, specify $(System.TeamProject)-$(Release.EnvironmentName). This variable maps the project and release name with your IoT Edge deployment ID.

    If your task involves using an image that resides in a private Docker Trusted Registry that isn't visible to the public cloud, you can set the SKIP_MODULE_IMAGE_VALIDATION environment variable to true to skip image validation.

    Add Azure IoT Edge tasks for your dev stage

  11. Select Save to save your changes to the new release pipeline. Return to the pipeline view by selecting Pipeline tab from the menu.

Note

Layered deployments are not yet supported in Azure IoT Edge tasks in Azure DevOps.

However, you can use an Azure CLI task in Azure DevOps to create your deployment as a layered deployment. For the Inline Script value, you can use the az iot edge deployment create command:

az iot edge deployment create -d {deployment_name} -n {hub_name} --content modules_content.json --layered true

Verify IoT Edge CI/CD with the build and release pipelines

To trigger a build job, you can either push a commit to source code repository or manually trigger it. In this section, you manually trigger the CI/CD pipeline to test that it works. Then verify that the deployment succeeds.

  1. From the left pane menu, select Pipelines and open the build pipeline that you created at the beginning of this article.

  2. You can trigger a build job in your build pipeline by selecting the Run pipeline button in the top right.

    Manually trigger your build pipeline using the Run pipeline button

  3. Review the Run pipeline settings. Then, select Run.

    Specify run pipeline options and select Run

  4. Select Agent job 1 to watch the run's progress. You can review the logs of the job's output by selecting the job.

    Review the job's log output

  5. If the build pipeline is completed successfully, it triggers a release to dev stage. The successful dev release creates IoT Edge deployment to target IoT Edge devices.

    Release to dev

  6. Click dev stage to see release logs.

    Release logs

  7. If your pipeline is failing, start by looking at the logs. You can view logs by navigating to the pipeline run summary and selecting the job and task. If a certain task is failing, check the logs for that task. For detailed instructions for configuring and using logs, see Review logs to diagnose pipeline issues.

Next steps