Edit

Integrate IoT Central with Azure Pipelines for continuous integration and continuous delivery

  • Article

Continuous integration and continuous delivery (CI/CD) refers to the process of developing and delivering software in short, frequent cycles using automation pipelines. This article shows you how to automate the build, test, and deployment of an IoT Central application configuration. This automation enables development teams to deliver reliable releases more frequently.

Continuous integration starts with a commit of your code to a branch in a source code repository. Each commit is merged with commits from other developers to ensure that no conflicts are introduced. Changes are further validated by creating a build and running automated tests against that build. This process ultimately results in an artifact, or deployment bundle, to deploy to a target environment. In this case, the target is an Azure IoT Central application.

Just as IoT Central is a part of your larger IoT solution, IoT Central is a part of your CI/CD pipeline. Your CI/CD pipeline should deploy your entire IoT solution and all configurations to each environment from development through to production:

Diagram that shows the stages of a typical CI/CD pipeline.

IoT Central is an application platform as a service that has different deployment requirements from platform as a service components. For IoT Central, you deploy configurations and device templates. These configurations and device templates are managed and integrated into your release pipeline by using APIs.

While it's possible to automate IoT Central app creation, you should create an app in each environment before you develop your CI/CD pipeline.

By using the Azure IoT Central REST API, you can integrate IoT Central app configurations into your release pipeline.

This guide walks you through the creation of a new pipeline that updates an IoT Central application based on configuration files managed in GitHub. This guide has specific instructions for integrating with Azure Pipelines, but could be adapted to include IoT Central in any release pipeline built using tools such as Tekton, Jenkins, GitLab, or GitHub Actions.

In this guide, you create a pipeline that only applies an IoT Central configuration to a single instance of an IoT Central application. You should integrate the steps into a larger pipeline that deploys your entire solution and promotes it from development to QA to preproduction to production, performing all necessary testing along the way.

The scripts currently don't transfer the following settings between IoT Central instances: dashboards, views, custom settings in device templates, pricing plan, UX customizations, application image, rules, scheduled jobs, saved jobs, and enrollment groups.

The scripts currently don't remove settings from the target IoT Central application that aren't present in the configuration file.

Prerequisites

You need the following prerequisites to complete the steps in this guide:

Download the sample code

To get started, fork the IoT Central CI/CD GitHub repository and then clone your fork to your local machine:

  1. To fork the GitHub repository, open the IoT Central CI/CD GitHub repository and select Fork.

  2. Clone your fork of the repository to your local machine by opening a console or bash window and running the following command.

    git clone https://github.com/{your GitHub username}/iot-central-CICD-sample

Create a service principal

While Azure Pipelines can integrate directly with a key vault, a pipeline needs a service principal for some dynamic key vault interactions such as fetching secrets for data export destinations.

To create a service principal scoped to your subscription:

  1. Run the following command to create a new service principal:

    az ad sp create-for-rbac -n DevOpsAccess --scopes /subscriptions/{your Azure subscription Id} --role Contributor

  2. Make a note of the password, appId, and tenant as you need these values later.

  3. Add the service principal password as a secret called SP-Password to your production key vault:

    az keyvault secret set --name SP-Password --vault-name {your production key vault name} --value {your service principal password}

  4. Give the service principal permission to read secrets from the key vault:

    az keyvault set-policy --name {your production key vault name} --secret-permissions get list --spn {the appId of the service principal}

Generate IoT Central API tokens

In this guide, your pipeline uses API tokens to interact with your IoT Central applications. It's also possible to use a service principal.

Note

IoT Central API tokens expire after one year.

Complete the following steps for both your development and production IoT Central apps.

  1. In your IoT Central app, select Permissions and then API tokens.

  2. Select New.

  3. Give the token a name, specify the top-level organization in your app, and set the role to App Administrator.

  4. Make a note of the API token from your development IoT Central application. You use it later when you run the IoTC-Config.ps1 script.

  5. Save the generated token from the production IoT Central application as a secret called API-Token to the production key vault:

    az keyvault secret set --name API-Token --vault-name {your production key vault name} --value '{your production app API token}'

Generate a configuration file

These steps produce a JSON configuration file for your development environment based on an existing IoT Central application. You also download all the existing device templates from the application.

  1. Run the following PowerShell 7 script in the local copy of the IoT Central CI/CD repository:

    cd .\iot-central-CICD-sample\PowerShell\
.\IoTC-Config.ps1

  2. Follow the instructions to sign in to your Azure account.

  3. After you sign in, the script displays the IoTC Config options menu. The script can generate a config file from an existing IoT Central application and apply a configuration to another IoT Central application.

  4. Select option 1 to generate a configuration file.

  5. Enter the necessary parameters and press Enter:

    • The API token you generated for your development IoT Central application.
    • The subdomain of your development IoT Central application.
    • Enter ..\Config\Dev as the folder to store the config file and device templates.
    • The name of your development key vault.

  6. The script creates a folder called IoTC Configuration in the Config\Dev folder in your local copy of the repository. This folder contains a configuration file and a folder called Device Models for all the device templates in your application.

Modify the configuration file

Now that you have a configuration file that represents the settings for your development IoT Central application instance, make any necessary changes before you apply this configuration to your production IoT Central application instance.

  1. Create a copy of the Dev folder created previously and call it Production.

  2. Open IoTC-Config.json in the Production folder using a text editor.

  3. The file has multiple sections. However, if your application doesn't use a particular setting, that section is omitted from the file:

    {
  "APITokens": {
    "value": [
      {
        "id": "dev-admin",
        "roles": [
          {
            "role": "ca310b8d-2f4a-44e0-a36e-957c202cd8d4"
          }
        ],
        "expiry": "2023-05-31T10:47:08.53Z"
      }
    ]
  },
  "data exports": {
    "value": [
      {
        "id": "5ad278d6-e22b-4749-803d-db1a8a2b8529",
        "displayName": "All telemetry to blob storage",
        "enabled": false,
        "source": "telemetry",
        "destinations": [
          {
            "id": "393adfc9-0ed8-45f4-aa29-25b5c96ecf63"
          }
        ],
        "status": "notStarted"
      }
    ]
  },
  "device groups": {
    "value": [
      {
        "id": "66f41d29-832d-4a12-9e9d-18932bee3141",
        "displayName": "MXCHIP Getting Started Guide - All devices"
      },
      {
        "id": "494dc749-0963-4ec1-89ff-e1de2228e750",
        "displayName": "RS40 Occupancy Sensor - All devices"
      },
      {
        "id": "dd87877d-9465-410b-947e-64167a7a1c39",
        "displayName": "Cascade 500 - All devices"
      },
      {
        "id": "91ceac5b-f98d-4df0-9ed6-5465854e7d9e",
        "displayName": "Simulated devices"
      }
    ]
  },
  "organizations": {
    "value": []
  },
  "roles": {
    "value": [
      {
        "id": "344138e9-8de4-4497-8c54-5237e96d6aaf",
        "displayName": "Builder"
      },
      {
        "id": "ca310b8d-2f4a-44e0-a36e-957c202cd8d4",
        "displayName": "Administrator"
      },
      {
        "id": "ae2c9854-393b-4f97-8c42-479d70ce626e",
        "displayName": "Operator"
      }
    ]
  },
  "destinations": {
    "value": [
      {
        "id": "393adfc9-0ed8-45f4-aa29-25b5c96ecf63",
        "displayName": "Blob destination",
        "type": "blobstorage@v1",
        "authorization": {
          "type": "connectionString",
          "connectionString": "DefaultEndpointsProtocol=https;AccountName=yourexportaccount;AccountKey=*****;EndpointSuffix=core.windows.net",
          "containerName": "dataexport"
        },
        "status": "waiting"
      }
    ]
  },
  "file uploads": {
    "connectionString": "FileUpload",
    "container": "fileupload",
    "sasTtl": "PT1H"
  },
  "jobs": {
    "value": []
  }
}

  4. If your application uses file uploads, the script creates a secret in your development key vault with the value shown in the connectionString property. Create a secret with the same name in your production key vault that contains the connection string for your production storage account. For example:

    az keyvault secret set --name FileUpload --vault-name {your production key vault name} --value '{your production storage account connection string}'

  5. If your application uses data exports, add secrets for the destinations to the production key vault. The config file doesn't contain any actual secrets for your destination, the secrets are stored in your key vault.

  6. Update the secrets in the config file with the name of the secret in your key vault.

    Destination type Property to change
    Service Bus queue connectionString
    Service Bus topic connectionString
    Azure Data Explorer clientSecret
    Azure Blob Storage connectionString
    Event Hubs connectionString
    Webhook No Auth N/A

    For example:

    "destinations": {
  "value": [
    {
      "id": "393adfc9-0ed8-45f4-aa29-25b5c96ecf63",
      "displayName": "Blob destination",
      "type": "blobstorage@v1",
      "authorization": {
        "type": "connectionString",
        "connectionString": "Storage-CS",
        "containerName": "dataexport"
      },
      "status": "waiting"
    }
  ]
}

  7. To upload the Configuration folder to your GitHub repository, run the following commands from the IoTC-CICD-howto folder.

     git add Config
 git commit -m "Adding config directories and files"
 git push

Create a pipeline

  1. Open your Azure DevOps organization in a web browser by going to https://dev.azure.com/{your DevOps organization}
  2. Select New project to create a new project.
  3. Give your project a name and optional description and then select Create.
  4. On the Welcome to the project page, select Pipelines and then Create Pipeline.
  5. Select GitHub as the location of your code.
  6. Select Authorize AzurePipelines to authorize Azure Pipelines to access your GitHub account.
  7. On the Select a repository page, select your fork of the IoT Central CI/CD GitHub repository.
  8. When prompted to log into GitHub and provide permission for Azure Pipelines to access the repository, select Approve & install.
  9. On the Configure your pipeline page, select Starter pipeline to get started. The azure-pipelines.yml is displayed for you to edit.

Create a variable group

An easy way to integrate key vault secrets into a pipeline is through variable groups. Use a variable group to ensure the right secrets are available to your deployment script. To create a variable group:

  1. Select Library in the Pipelines section of the menu on the left.

  2. Select + Variable group.

  3. Enter keyvault as the name for your variable group.

  4. Enable the toggle to link secrets from an Azure key vault.

  5. Select your Azure subscription and authorize it. Then select your production key vault name.

  6. Select Add to start adding variables to the group.

  7. Add the following secrets:

    • The IoT Central API Key for your production app. You called this secret API-Token when you created it.
    • The password for the service principal you created previously. You called this secret SP-Password when you created it.

  8. Select OK.

  9. Select Save to save the variable group.

Configure your pipeline

Now configure the pipeline to push configuration changes to your IoT Central application:

  1. Select Pipelines in the Pipelines section of the menu on the left.

  2. Replace the contents of your pipeline YAML with the following YAML. The configuration assumes your production key vault contains:

    • The API token for your production IoT Central app in a secret called API-Token.
    • Your service principal password in a secret called SP-Password.

    Replace the values for -AppName and -KeyVault with the appropriate values for your production instances.

    You made a note of the -AppId and -TenantId when you created your service principal.

    trigger:
- master
variables:
- group: keyvault
- name: buildConfiguration
  value: 'Release'
steps:
- task: PowerShell@2
  displayName: 'IoT Central'
  inputs:
    filePath: 'PowerShell/IoTC-Task.ps1'
    arguments: '-ApiToken "$(API-Token)" -ConfigPath "Config/Production/IoTC Configuration" -AppName "{your production IoT Central app name}" -ServicePrincipalPassword (ConvertTo-SecureString "$(SP-Password)" -AsPlainText -Force) -AppId "{your service principal app id}" -KeyVault "{your production key vault name}" -TenantId "{your tenant id}"'
    pwsh: true
    failOnStderr:  true

  3. Select Save and run.

  4. The YAML file is saved to your GitHub repository, so you need to provide a commit message and then select Save and run again.

Your pipeline is queued. It may take a few minutes before it runs.

The first time you run your pipeline, you're prompted to give permissions for the pipeline to access your subscription and to access your key vault. Select Permit and then Permit again for each resource.

When your pipeline job completes successfully, sign in to your production IoT Central application and verify the configuration was applied as expected.

Promote changes from development to production

Now that you have a working pipeline you can manage your IoT Central instances directly by using configuration changes. You can upload new device templates into the Device Models folder and make changes directly to the configuration file. This approach lets you treat your IoT Central application's configuration the same as any other code.

Next step

Now that you know how to integrate IoT Central configurations into your CI/CD pipelines, a suggested next step is to learn how to Manage and monitor IoT Central applications.