Add continuous integration to your container builds

Article
08/08/2023

Continuous integration is a software development process in which an application is kept in a continually releasable state by providing automated builds with each commit to a specific code base. You can add continuous integration to virtually any build system, but two that are particularly convenient are GitHub Actions and Azure Pipelines. In this topic, you will see how to use either GitHub Actions or Azure Pipelines to automate the Docker build steps described in Use containers to build Azure Sphere apps.

Use GitHub Actions to automatically build your container

GitHub Actions allow you to automate your build process directly from your GitHub repositories. Thus, the first step in using GitHub Actions is to create or open a GitHub repository that contains your application code. This topic assumes you have created a GitHub repository containing the Blink application generated in Tutorial: Build a high-level application and that your project is named "Blink". As with any continuous integration project, make sure that your project builds locally and provides the expected artifacts before you attempt to automate the process. In this example, we assume that after a successful build, the out directory contains a Blink.imagepackage file.

In the top-level directory of your GitHub repository, create a directory named .devcontainer, and create a file named Dockerfile in that directory with the following content:

FROM mcr.microsoft.com/azurespheresdk:latest AS dev

FROM dev AS build
COPY ./ /src/
WORKDIR /out
RUN cmake -G "Ninja" -DCMAKE_TOOLCHAIN_FILE="/opt/azurespheresdk/CMakeFiles/AzureSphereToolchain.cmake" \
    -DAZURE_SPHERE_TARGET_API_SET="latest-lts" -DCMAKE_BUILD_TYPE="Release" "/src"
ENTRYPOINT [ "ninja" ]

The initial FROM line specifies the standard Azure Sphere Docker image as the base development container, and the second says to use that base container as the build environment. The COPY line copies the contents of the repository into the container's /src/ directory. The WORKDIR specifies the build directory. The RUN command provides the CMake command to generate the build files. Finally, the ENTRYPOINT specifies that ninja should be invoked to actually build the application.

In the top-level directory of your repository, create the .github/workflows directory and add a file named ci.yml with the following content:

# This is a basic workflow to help you get started with Actions

name: ContinuousIntegration

# Controls when the action will run. Triggers the workflow on push or pull request
# events, but including workflow_dispatch also allows manual execution
on:
  push:
  pull_request:
  workflow_dispatch:


jobs:
  build:
    runs-on: ubuntu-latest
    name: Build Azure Sphere Apps
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    - name: Build image for az sphere builds and Start container from build image
      run: |
        docker build --target build -t hlbuildimage -f .devcontainer/Dockerfile .
        docker run --name hlbuildcontainer hlbuildimage
    - name: Copy container build output
      run:
        docker cp hlbuildcontainer:/out HLOutput
    - name: Publish HL imagepackage
      uses: actions/upload-artifact@v2
      with:
        name: HL imagepackage
        path: ${{ github.workspace }}/HLOutput/Blink.imagepackage

This workflow has only one job—to build the application; the job runs on a GitHub Actions runner, in this case ubuntu-latest, and has four steps:

Step 1, Checkout, is a standard GitHub action that simply checks out your repository to the ubuntu-latest runner.
Step 2 builds the image (docker build) and starts the container (docker run).
Step 3 copies the output from the container to the runner.
Step 4, Publish HL imagepackage, publishes the high-level application imagepackage as an artifact.

Commit these changes to your main branch and select Actions. You should now see a page labeled "All workflows", with at least one workflow running or completed. If the workflow completes successfully, a green check mark appears beside it. Click on a successful workflow and you should see a box labeled "Artifacts" containing one artifact labeled "HL imagepackage". Download this artifact and unpack the imagepackage file; you can then create a deployment or sideload the application to your device.

Use Azure Pipelines to automatically build your container

Azure Pipelines allow you to automate your build process directly from your GitHub repositories (and many other code repositories as well). This topic assumes you already belong to an organization with an Azure DevOps project and have access to Azure Pipelines. The first step in using Azure Pipelines is to create or open a repository that contains your application code. This topic assumes you have created a GitHub repository containing the Blink application generated in Tutorial: Build a high-level application.

In the top-level directory of this repository, create the .devcontainer directory, and create a Dockerfile file in that directory with the following content:

FROM mcr.microsoft.com/azurespheresdk:latest AS dev

FROM dev AS build
COPY ./ /src/
WORKDIR /out
RUN cmake -G "Ninja" -DCMAKE_TOOLCHAIN_FILE="/opt/azurespheresdk/CMakeFiles/AzureSphereToolchain.cmake" \
    -DAZURE_SPHERE_TARGET_API_SET="latest-lts" -DCMAKE_BUILD_TYPE="Release" "/src"
ENTRYPOINT [ "ninja" ]

To create the pipeline:

Log in to your Azure DevOps project and open Pipelines.
Select New Pipeline, then select GitHub when asked Where is your code? You may be taken to a GitHub authentication page; complete the authentication and continue to the page to select your repository.
Select your Blink repository. You are taken to a page titled Configure your pipeline.
Select Starter pipeline. This opens a file named azure-pipelines.yml in the top-level directory of your repository with a Hello, World task.
Select Save and run. Accept the default commit message and again select Save and run. The azure-pipelines.yml file is committed to your GitHub repository and the pipeline is created.

Replace the contents of the azure-pipelines.yml file with the following content:

# Docker
# Build a Docker image
# /azure/devops/pipelines/languages/docker

trigger:
- main

resources:
- repo: self

variables:
  tag: '$(Build.BuildId)'

stages:
- stage: Build
  displayName: Build image
  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - bash: docker build --target build -t hlbuildimage -f .devcontainer/Dockerfile . &&  docker run --name hlbuildcontainer hlbuildimage && docker cp hlbuildcontainer:/out $(Build.ArtifactStagingDirectory)/HLOutput
      displayName: Build high-level Azure Sphere application in a container and copy the output
    - task: PublishBuildArtifacts@1
      displayName: Publish build artifacts
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)/HLOutput/Blink.imagepackage'
        ArtifactName: 'BlinkSample.imagepackage'
        publishLocation: 'Container'

This workflow has only one job—to build the application; the job runs on a Azure DevOps agent, in this case ubuntu-latest, and has two steps:

Step 1 builds the image (docker build), starts the container (docker run), and copies the output from the container to the agent.
Step 2, Publish build artifacts, publishes the high-level application imagepackage as an artifact.

Commit these changes to your main branch. In Azure DevOps, open Pipelines again. You should see a run of your pipeline in progress or just completed. If the run shows a green checkmark, the build was successful. Select the successful run; you should see 1 Published in the Related column. Download this artifact and unzip the imagepackage file; you can then create a deployment or sideload the application to your device.

Add continuous integration to Azure Sphere sample applications

GitHub Actions and Azure Pipelines are meant to automate builds for a single project, such as those downloaded from the Microsoft samples browser. The Azure Sphere Samples on GitHub are a collection of projects with some shared resources. To use one of these samples in continuous integration, you need to incorporate any needed shared resources. Usually, this means at least creating a HardwareDefinitions directory in the top-level directory of your project and editing the CMakeLists.txt file to point to the local copy. For example, if you create a project based on the HelloWorld/HelloWorld_HighLevelApp sample, the top-level directory initially looks like this:

.vscode
.gitignore
applibs_versions.h
app_manifest.json
CMakeLists.txt
CMakeSettings.json
launch.vs.json
LICENSE.txt
main.c
README.md

The CMakeLists.txt file contains the following line pointing to the shared HardwareDefinitions directory in the Samples repo:

azsphere_target_hardware_definition(${PROJECT_NAME} TARGET_DIRECTORY "../../../HardwareDefinitions/mt3620_rdb" TARGET_DEFINITION "sample_appliance.json")

To enable your project to build, copy the HardwareDefinitions folder into your top-level directory, then edit the CMakeLists.txt file to use the local location:

azsphere_target_hardware_definition(${PROJECT_NAME} TARGET_DIRECTORY "HardwareDefinitions/mt3620_rdb" TARGET_DEFINITION "sample_appliance.json")

Again, verify that your project builds locally before you attempt to automate with GitHub Actions.

Add continuous integration to your container builds

Use GitHub Actions to automatically build your container

Use Azure Pipelines to automatically build your container

Add continuous integration to Azure Sphere sample applications

Feedback

Additional resources