Set up a CI/CD pipeline with unit testing and code coverage for your VSTS extension
We receive a notification that there’s a pull request to review for our Team Services extensibility project. We discuss the proposed changes as a team, eventually merge the new feature(s) into the master branch, and close the pull request. Done … right?
Not quite. The merge triggers our automated CI/CD pipeline, which consistently builds, tests and delivers our package to the Visual Studio marketplace.
This post should be a valuable reference, if you’re interested in how we configured our Countdown Widget pipeline and how the team included testing and code coverage. Let’s walk through the configuration of our pipeline.
MODIFIED 2017.08.14 – Update to align with latest codebase from https://github.com/ALM-Rangers/Countdown-Widget-Extension. If you remember the previous pipeline, you’ll notice that the pipeline has been simplified by code generated by the the fastest path to a new VSTS extension generator.
Build
We start with the continuous integration build, made up of thirteen steps.
The magic around unit testing and code coverage happens during steps 8-11. See Testing a Team Services extension and tracking code coverage for more details.
# | BUILD STEP | BUILD SNIPPET | WHAT’S HAPPENING? |
1 | Install npm dependencies | Install all modules listed as dependencies in package.json. | |
2 | npm custom | Run a custom npm task (install –g whitesource) to install the Whitesource npm plugin. | |
3 | Replace tokens in Whitesource config | We replace the WhiteSourceAPIKey in the whitesource.config.json file with the value of the variable with the same name. Token Regex is “__(\w+)__”, which means that we’re looking for one or more letter, digit or underscore characters, surrounded by a double underscore “__name__”. |
|
4 | Replace tokens for TelemetryClient | We replace the INSTRUMENTATION key in the TelemetryClientSettings.ts file. Variables
|
|
5 | Run whitesource | Run the Whitesource tool to scan and detect security vulnerabilities, problematic OSS licenses and quality issues. See Manage your open source usage and security in your pipeline for details. | |
6 | npm run build | Run the package build script, as defined under “scripts” in package.json. | |
7 | nom run tests | Run the package test script, as defined under “scripts” in package.json. | |
8 | publish test results | Publish the test results, located in the ./reportTests/ folder. | |
9 | Publish Code Coverage Results | Publish the code coverage results from the formatted cobertura-coverage.xml file. | |
10 | Package Extension | We package the extension using the VSTS Developer Tools Build Tasks extension and generate the output.vsix artifact. Extension ID is appended with Alpha to differentiate it from deployed DEV, BETA, and PROD packages. |
|
11 | Publish Artifact | We publish the artifact, output.vsix, to $(Extension.OutputPath) to be picked up by the release. |
Release
Next we continue with continuous delivery (CD), made up of three environments. With the exception of configuration values, this part of the pipeline is identical (consistent) to the one we covered in up Set up a CI/CD pipeline for your Team Services extension.
Continuous Delivery is the ability to use the output from the CI to build and deploy the new known good build to one or more environments automatically ( bit.ly/28PWsfk ). There is a subtle difference between Continuous Delivery and Continuous Deployment. The latter is to a single environment. A small team might only implement Continuous Deployment because each change goes directly to production. Continuous Delivery is moving code through several environments, ultimately ending up in production, which may include automated UI, load and performance tests and approvals along the way. – Extract from DevOps - Applying DevOps to a Software Development Project
DEV
The environment with one task is triggered after a successful release creation. Both the pre-deployment and post-deployment approvals are automatic, ensuring that the development team is always working with the latest version. There’s no manual intervention.
# | RELEASE STEP | BUILD SNIPPET | WHAT’S HAPPENING? |
1 | Publish Extension | We publish the extension to the alm-rangers publisher, using the VSTS Developer Tools Build Tasks extension.
|
BETA
The environment with only one task is triggered after a successful deployment to the DEV environment. The BETA approvals mandate that one of the specified users approve the release, to ensure that a high quality bar can be met. Users typically include the project lead and program manager.
# | RELEASE STEP | BUILD SNIPPET | WHAT’S HAPPENING? |
1 | Publish Extension | We publish the extension to the alm-rangers publisher, using the VSTS Developer Tools Build Tasks extension.
|
PROD
The environment with one task is triggered after a successful deployment to the BETA environment. The PROD approvals mandate that all of the specified users approve the release, to validate that the aka.ms/vsarDoD (definition of done) is met. Users typically include the project lead, program manager, and product owner.
# | BUILD STEP | BUILD SNIPPET | WHAT’S HAPPENING? |
1 | Publish Extension | We publish the extension to the ms-devlabs publisher, using the VSTS Developer Tools Build Tasks extension.
|
That’s it!
Now that you have an understanding of how our pipeline works, you should explore how you can configure your continuous integration and delivery pipeline by #RubDevOpsOnIt.
Other pipelines examples
Reference Information
DevOps - Applying DevOps to a Software Development Project, lcov-to-cobertura-xml, Set up a CI/CD pipeline for your Team Services extension, Testing a Team Services extension and tracking code coverage, VSTS Developer Tools Build Tasks
Comments
- Anonymous
May 08, 2018
Are you aware of any working tutorials or examples of how to unit test a vss-sdk web extension that uses amd module and webpack and typescript? I've spent days trying various tutorials and I always end up hitting a wall with errors such as 'define is not defined'.