Set up continuous integration and continuous delivery for a Composer bot

APPLIES TO: Composer v2.x

You can set up continuous integration and continuous delivery (CI/CD) to deploy new versions of Bot Framework Composer bots. This article describes how to do so using Composer, Azure DevOps, and git.

Tip

You need to provision the resources for your bot first, before setting up CI/CD.

Prerequisites

The sample YAML and PowerShell files referenced in this article can be found on GitHub at microsoft/BotBuilder-Samples.

File Description
buildAndDeploy.yaml The main YAML file used when the Azure DevOps pipeline is triggered. It maps Azure DevOps variables (see Define variables for additional information) into YAML runtime parameters and then sequentially calls the YAML templates in the templates folder to build and deploy the bot.
templates/installPrerequisites.yaml Installs the tools needed to run the pipeline, like npm, BF CLI, and .NET core.
templates/buildAndDeployModels.yaml Builds, trains, and deploys LUIS and QnA Maker models. This template also creates a generated subfolder in your bot project's directory. The generated folder is needed by the bot.
templates/buildAndDeployDotNetWebApp.yaml Builds the dotnet bot app, prepares the zip package, and deploys it to Azure. It also configures the app settings for the app in Azure once it is deployed.
templates/Build-LUIS.ps1 Builds and trains LUIS models.
templates/Build-Orchestrator.ps1 Builds and trains Orchestrator models.
templates/LUUtils.ps1 Contains language understanding functions used in other scripts.

Provision your Azure environment

You first need to provision Azure resources to publish your bot. Take note of the configuration settings so you can later configure the CI/CD pipeline. To make this easier, copy and paste the Pipeline parameters table provided in a separate document and use it as a template.

Add YAML files to your bot project source code

Download the sample YAML and PowerShell files provided, and add them to your bot's source code in git under the build folder. Then merge the changes into your main branch.

YAML source

Tip

This is a good time to update the BotProjectDirectory and BotProjectName settings in your Pipeline parameters table with your .csproj name and the relative path to its location in your source tree.

Create and configure your CI/CD pipeline

Configure an Azure Service Connection for your Azure DevOps project

Important

Ensure that your user has 'Owner' or 'User Access Administrator' permissions on the Azure Subscription before you try to create the service connection.

Your pipeline uses a service connection to interact with the resource group where the Azure resources for your bot are located. Create a service connection and note the name in the Pipeline parameters table under the AzureServiceConnection setting.

To configure a Service Connection, go to the Project Settings section in Azure DevOps, and select Service connections, as seen below:

Select Service connections

Then select the New Service Connection button and create an Azure Resource Manager resource:

New service connection - ARM

Select Service principal (automatic) and grant it permissions to the Azure resource group where you created your bot resources.

New service connection - Service principal (automatic

See Service connections in the Azure DevOps documentation for additional information.

Create the pipeline

In your Azure DevOps project pipelines section, select new pipeline.

Select your source code provider and repository in the first two tabs and in the Configure steps above. Scroll down to the bottom of the page and select Existing Azure Pipelines YAML file:

Create existing Azure Pipelines YAML

Then select the buildAndDeploy.yaml file you previously added to the repository:

Select existing YAML file

In the Review tab, select the Variables button and create the variables with the values defined in the Pipeline parameters table. Make sure you check the Keep this value secret option for sensitive parameters.

Pipeline variables example

Then select Run. This saves and executes your pipeline for the fist time, seen below:

Run pipeline

Configure your LUIS prediction resources (first time only)

Depending on your bot, the pipeline will create one or more LUIS apps when you run it for the first time. Once that is done, make sure you set the prediction resource in the Manage section for your LUIS applications:

Configure LUIS

Run and test your deployed bot

You can now run and test your bot from the configured channels.

Additional information

Pipeline parameters

Configuration setting Description Example
AzureServiceConnection The name of the Azure DevOps service connection used to deploy the bot to Azure MyCICDBotDeployment
BotName Your bot's name MyCICDBot
BotProjectDirectory The relative path from the repo root to your bot's .csproj file BasicAssistant/BasicAssistant
BotProjectName The name of your bot's .csproj file BasicAssistant.csproj
BuildYamlDirectory The relative path from the repo root to your pipeline's .yaml files build/yaml
LuisAuthoringKey The authoring key for your LUIS authoring resource (from the LUIS portal <conversation_app>/Manage/Azure Resources/Authoring Resource/Primary Key) 1234567890abcdef1234567890abcdef
LuisAuthoringRegion The authoring region for your LUIS authoring resource (from the LUIS portal <conversation_app>/Manage/Azure Resources/Authoring Resource/Location) westus
LuisEndpoint The endpoint URL for your LUIS prediction resource (from the LUIS portal <conversation_app>/Manage/Azure Resources/Prediction Resources/Endpoint URL) https://westus.api.cognitive.microsoft.com
LuisEndpointKey The endpoint key for your LUIS prediction resource (from the LUIS portal <conversation_app>/Manage/Azure Resources/Prediction Resources/Primary Key) 1234567890abcdef1234567890abcdef
MicrosoftAppId The Azure Active Directory app registration app ID for your bot 11111111-1111-1111-1111-111111111111
MicrosoftAppPassword The Azure Active Directory app registration app password for your bot YourAppPassword
QnAEndpointKey The endpoint key used to access QnA Maker (from QnA portal <your_knowledgebase>/Settings/Deployment Details/Authorization: EndpointKey) 11111111-1111-1111-1111-111111111111
QnAHostName Your QnA Maker host URL (from QnA portal <your_knowledgebase>/Settings/Deployment Details/Host) https://mycicdbotapp-qna-qnahost.azurewebsites.net/qnamaker https://mycicdbotapp-qna-qnahost.azurewebsites.net/qnamaker
QnAKnowledgebaseId Your QnA Maker knowledge base ID (from QnA portal <your_knowledgebase>/Settings/Deployment Details/knowledgebases) 11111111-1111-1111-1111-111111111111
QnARegion Your QnA Maker region (from the Azure portal <qna_maker_resource>/Keys and Endpoint/Location) westus
QnASubscriptionKey The QnA Maker subscription key (from the Azure portal <qna_maker_resource>/Keys and Endpoint/KEY 1) 1234567890abcdef1234567890abcdef
ResourceGroupName The Azure resource group containing your bot's deployed resources MyTestBot-RG
WebAppName The Azure app service name for your bot MyCICDBotApp

Here is an example of how the pipeline variables would look in Azure DevOps:

Pipeline variables list

Portals

  • Azure portal - Access to your bot's deployment and Azure resources.
  • LUIS portal - Most LUIS pipeline parameter values are under <app_name>/Manage/Azure Resources.
  • QnA Maker portal - Most QnA pipeline parameter values are under <your_knowledgebase>/Settings/Deployment Details.

LUIS troubleshooting

If after running a pipeline deployment, Test in Web Chat returns "403 (Forbidden)" or similar, you may have a mismatch of LUIS App ID, LUIS Endpoint, or LUIS Endpoint key.

  1. Look up the values in the Azure portal, in your App Service:

    • LUIS App ID is in the App Service file wwwroot/generated/luis.settings.composer.*.json.
    • Endpoint and endpoint key are in the App Service's Configuration/Application settings under the names luis__endpoint and luis__endpointKey. (These values override anything found in settings/appsettings.json.)
  2. Look up the same values in the LUIS portal, in your conversation app:

    • LUIS App ID is in Manage/Settings/Application Settings.
    • Endpoint and endpoint key are in Manage/Azure Resources/Prediction Resources under the names Endpoint URL and Primary Key.
  3. Verify that the 3 values in Azure match those same values found in LUIS.

Tip

For testing, you may use the Endpoint URL and Primary Key values from Authoring Resource instead of the ones from Prediction Resources.

Submit Feedback or file an issue

  1. Go to the create new issue page for the BotFramework-Composer repository.
  2. Select the appropriate option: Bug, Feature Request, or Other.
  3. Fill out template and submit your issue.
  4. Add the Area: CICD label to the issue.