What do you think of our definition of done (DoD) for Extensions? (aka.ms/vsarDoD)

  • Article

The Scrum Guide recommends that when anyone states “Done”, everybody in the team understands what “Done” means. Furthermore, when there are multiple teams working on the same product, the teams must mutually agree on the Definition of Done (DoD).

WORK IN PROGRESS v1.0-2017.04.05

This post will evolve over time and shares our definition of done (DoD) for the DevLabs extension projects we are building for Visual Studio Team Services. You are welcome to re-use the DoD “as is” or adapt it to suit your needs.

Microsoft DevLabs is an outlet for experiments from Microsoft, experiments that represent some of the latest ideas around developer tools. Solutions in this category are designed for broad usage, and you are encouraged to use and provide feedback on them; however, these extensions are not supported nor are any commitments made as to their longevity.

Definition of Done (DoD) for our extension pipeline

image

       
APPROVAL BETA Beta deployment must be pre-approved by the project lead or program manager. [ ]
PROD Prod deployment must be pre-approved by the project lead and program manager. [ ]
PIPELINE Continuous Integration (CI)

Continuous integration build created in VS Team Service. See Build practices and “Embracing DevOps when building Visual Studio Team Services Extensions” article for more details.

 [ ]
Continuous Deployment (CD)

Continuous deployment release created in VS Team Services. See Release practices and “Embracing DevOps when building Visual Studio Team Services Extensions” article for more details

 [ ]
QUALITY Bugs No known critical or high priority bugs. [ ]
Impediments No known priority 1 impediments. [ ]
OSS OSS request approved for solution if open sourcing the code.OSS request approval for 3rd party OSS artefacts include with the solution. [ ][ ]
Performance Reviewed solution with the TEST SMEs and Test Guidelines are met. [ ]
User Experience Reviewed solution with the UX SMEs and UX Guidelines are met. [ ]
PoliCheck All issues from code and documentation scan for sensitive terminology fixed. [ ]
Test Plan Create (clone) a test plan for each major release and revision. [ ]
Test Cases Create test cases to validate deliverables and guide exploratory testing. [ ]
SOLUTION Extension manifest

Recommended manifest settings:

- Set version to 0.0.0- Set publisher to an empty string. - Set ID to extension name, without spaces. - Mark the extension as private (public: false). - Remove all galleryFlags.

 [ ][ ][ ][ ][ ]
Metrics Application Insights (AI) resource created on Azure using extension name.Unique instrumentation key per extension | service.Code instrumented as outlines under Application Insights practices. [ ][ ][ ]
Marketplace description Crisp overview with visuals. Quick steps to get started.Learn more section with: - Link to OSS repository, if applicable. - Link to support twitter.com/almrangers. - DevLabs notice if publishing to ms-devlabs publisher. - Active contributors as confirmed by the project lead (PL). - Feedback secion Optionally add a 1-2min demo video. [ ][ ][ ]

[ ]
Source files All source files have a header, aligned with OSS request. See example below.License matches OSS Request approval. [ ][ ]

 

Source header sample

Option 1 – if you do not have a license file in your project

//---------------------------------------------------------------------
// <copyright file="name of this file, i.e. FolderManager.ts">
// This code is licensed under the MIT License.
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
// ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
// TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
// PARTICULAR PURPOSE AND NONINFRINGEMENT.
// </copyright>
// <summary>
// …add a crisp summary here…
// </summary>
//---------------------------------------------------------------------

Option 2 – if you have a license file you can reference in your project

//---------------------------------------------------------------------------------------------
// <copyright file="name of this file, i.e. FolderManager.ts">
// Licensed under the MIT License. See License.txt in the project root for license information.
// </copyright>
// <summary>
// …add a crisp summary here…
// </summary>
//---------------------------------------------------------------------------------------------

Practices

Collection of common practices, cross-referenced by our Definition of Done (DoD).

Application Insights

  • Add a NuGet reference to Application Insights
    clip_image001
  • Add telemetry client class and update the instrumentation key
  • Include imported AI scripts in the VSIX
  • Ensure the VSS SDK is ready before calling the Init method

Build

  • Name: vsarVSTS SOLUTION NAME, example vsarVSTS Folder Management
  • Trigger: Continuous Integration (CI)
  • Output build output to a separate folder
  • Verify that build runs successfully
  • Verify that a valid VSIX is produced as output

Release

  • Name of release matches name of build
  • Link previously created Build Definition as an artefact source
  • Tag extension ID with the environment for Dev and Beta.
    • Dev environment: FolderManagementDev
    • Beta environment: FolderManagementBeta
    • Prod environment: NO suffix!
  • Create three environments:
    • Dev uses team publisher, private extension, and shared with DEV sandbox
    • Beta uses alm-rangers publisher, private extension, and is shared UAT sandbox
    • Prod uses ms-devlabs publisher, public extension and is NOT shared
  • Create/re-use three marketplace connections with access token for relevant environment

Test / Performance

FYI - Testing a Team Services extension and tracking code coverage

  • All automated tests written, executed, passed and added to the main regression pack
  • All functional tests written, executed and passed
  • All non-functional tests written, executed and passed
  • All regression tests copied to the master regression test plan
  • All unit tests written, executed, passed and included in CI build
  • Build Verification Test (BVT) pack updated
  • Test coverage should target 70-80%

User Experience

TO BE DEFINED.

Post-Release Maintenance

  • Tag release to be retained indefinitely.
  • Tag build to be retained indefinitely.
  • Announce release on team blog.

Change log

  • 2016.12.13
    • Add reference to Testing a Team Services extension and tracking code coverage blog
  • 2016.06.01
    • Change references to Public environment to Prod for consistency.
    • Remove “PROD deployment must be post-approved by the program manager", which is implicit.

Feedback?

Add a comment below or ping us here.