Article
01/04/2019

May 2018

Volume 33 Number 5

[DevOps]

Any Language, Any Platform with Azure DevOps Projects

By Willy-Peter Schaub, Alex Mullans | May 2018

Over the past two years, Microsoft has rolled out an ambitious continuous integration (CI) and continuous delivery (CD) pipeline initiative for its DevOps platform. The ALM | DevOps Rangers team has been active in the effort, covering the creation of and learnings from each release, both in numerous blog posts and in MSDN Magazine articles such as “Applying DevOps to a Software Development Project” (msdn.com/magazine/mt767695) and “The Road to Continuous Delivery with Visual Studio Team Services” (msdn.com/magazine/mt814804). Years of investment are paying dividends, as Microsoft today boasts a fast-paced and heterogeneous DevOps ecosystem built around a Visual Studio Team Services (VSTS) platform that has evolved by leaps and bounds.

Behind this effort are development teams that demand built-in support for popular application frameworks, automatic and seamless CI/CD pipeline integration, and built-in monitoring support—all in the language and platform of their choice. In addition, developers want to build any application, on any Azure service, in minutes. Speed and simplicity are paramount.

Microsoft Azure DevOps Projects (bit.ly/2E5zkWG) addresses all these needs. It enables developers to launch an app on any Azure App Service in just a few quick steps, providing everything needed to develop, deploy and monitor an app. Create a DevOps Project, and it provisions all the Azure resources and provides a Git code repository, Application Insights integration and a continuous delivery pipeline setup for deployment to Azure. The DevOps Project dashboard lets you monitor code commits, builds and deployments from a single view in the Azure portal.

This article explores how DevOps Projects enable cloud-powered CI/CD using VSTS to:

Get up and running with a new application and a full DevOps pipeline in just a few minutes
Support a wide range of popular frameworks such as .NET, Java, PHP, Node.js and Python
Integrate with built-in Application Insights for instant analytics and actionable insights
Start fresh or bring an existing application over from GitHub

Alternatively, we could look at the DevOps Project as a feature to help users “get started” on Azure. Imagine a developer who needs to develop and deploy a Node.js solution to Azure. The user needs to find which Azure service works best for the language, framework and platform scenario, prototype the environment, and finally create an engineering process for the team. Each step has a learning curve and multiple failure points. The DevOps Project simplifies all this and makes it failure-proof. It’s not just about DevOps, but also about making it simpler to get started on Azure.

Discussing DevOps

I’d like to pause for a moment to reiterate our definition of DevOps here at Microsoft. As Donovan Brown, principal DevOps manager in the Azure group describes it: “DevOps is the union of people, process and products to enable continuous delivery of value to our end users.”

You can also peruse the DevOps at Microsoft site (aka.ms/devops) for insight into the lessons learned by the Microsoft Developer Division during our seven-year journey to Agile DevOps. That effort has produced a culture here that’s focused on the customer, puts production first and continuously delivers value on a three-week cloud cadence.

In this article we’ll focus on the easier part of our DevOps definition, the products. As mentioned, developers crave a process that’s easy, quick and consistent. They demand support for their choice of programming language and platform, as well as for automation and out-of-the-box CI/CD pipelines.

As shown in Figure 1, a CI/CD pipeline is a sequence of distinct practices used to continuously integrate, test and deploy your changes to production. Insights from telemetry, user feedback, and live site incidents flow back to inform decision making, improve future releases, and most important, evaluate the hypothesis that motivated the deployment. It’s about continuously delivering value.

Figure1 DevOps Practices

Azure DevOps Projects

With Azure DevOps Projects, you can build an Azure application, on an Azure service, in minutes. You also get automatic full CI/CD pipeline integration, built-in monitoring and deployment to the platform of your choice. As shown in the table in Figure 2, there are several application, framework and deployment types to choose from when creating a new DevOps Project. It’s important to note that this table reflects information at the time of writing and that built-in support will have improved by the time you read this.

Figure 2 Application, Framework and Deployment Options

Application	Framework	Windows Deployment		Linux Deployment
		Web App	Virtual Machines	Web App	Web App for Containers
.NET	ASP.NET	X	X
.NET	ASP.NET Core	X	X	X	X
Java	Sprint	X			X
Java	JSF	X			X
Node.js	Express.js	X			X
	Sails.js				X
	Simple Node.js	X			X
PHP	Laravel	X			X
PHP	Simple PHP	X			X
Python	Django	X		X
	Bottle	X		X
	Flask	X		X	X

Let’s explore how to create a new DevOps Project on the Azure portal by creating a pair of applications. I recommend that you get started with Azure DevOps Projects and follow along using your own Azure and VSTS accounts. This approach will let you explore the resulting solutions in detail.

As shown in Figure3, the first application example is based on Java and the Spring framework, and has been deployed as a Web app on Windows. It’s one of many possible examples that shows a mix of technologies running on Windows—a common scenario for developers.

Figure 3 Java, JSF, Web App on Windows Application

In the Azure portal you create a new DevOps Project. You can then choose from a set of sample apps developed using .NET, Node.js, PHP, Python or Java, or you can bring our own code.

To deploy the application, (1) select the Java sample application, (2) choose Spring as your application framework and (3) select Web App on Windows as your Azure Service to deploy the app. Next, (4) enter a new project name and confirm or change the Azure and VSTS details. You’ll create a new VSTS account and change the App Service location to Canada Centre. Similarly, you can change Azure settings like subscription, pricing plan, Web App name and the location of Web App. Built-in validations ensure that the right values are entered and that downstream failures are avoided.

Once complete, you repeat the exercise, but this time select .NET, .NET Core Framework and the Web App for Containers on Linux to deploy the application. This is another simple example that shows how a mix of technologies can run in a container on Linux. By completing these four simple steps, you’ve created two DevOps projects, as shown in Figure 4.

Figure 4 DevOps Projects

These projects let you easily and quickly create complete CI/CD pipelines for several popular languages, frameworks and platforms. This is a useful asset for quick demonstrations and experimental projects, and is a great way to get your development team started with a comprehensive, extendible, and consistent process.

Magic Behind the Scenes

Obviously, none of this happened by magic. When you clicked the Done button after selecting the runtime framework, service and configuration, it kicked off a sequence of distinct stages, shown in Figure5. You can navigate to the shown DevOps Project Dashboard by selecting the new project in the list of DevOps projects or using the notifications on the Azure portal.

Figure 5 Pipeline View on Azure

The four stages involved are:

Azure resources: Created and configured your Azure App Service resources, App Service plan, and application endpoint.
Repository: Created a distributed Git repository and checked in sample code.
CI/CD Pipeline: Seamlessly connected with the VSTS collaboration solution for engineering teams, allowing you to plan, test, release and monitor your solutions.
Application Insights: Created and configured your Application Insights telemetry. This tool enables active monitoring and learning to proactively detect issues and continuously analyze and test hypotheses without code.

Note that the DevOps Project stages not only created and configured a CI/CD Pipeline(see section 3 in Figure 5), they also performed a pull of your Code, a continuous integration Build and deployed the release to the Dev environment.

From the DevOps Project dashboard, monitor your CI/CD pipelines, Application Insights telemetry, and navigate to the relevant Azure or VSTS resources.

When you navigate to your new VSTS account, you’ll find two new Projects: JaSpWeADemo and dNCoWe4DDemo (Figure 6). These are linked to the two Azure DevOps Projects.

Figure 6 Visual StudioTeam Services Account and Team Projects

In the Build, CI runs a Maven-based build whenever someone checks in code. To reduce the number of running builds, changes are batched when a build is running. You can fine-tune the events that trigger a build, for example specifying branches or using path filters to reduce the set of files that can trigger action.

When you explore the JaSpWeADemo project Pipeline, as shown in Figure 7, you’re presented with (1) Artifacts and (2) Environments. The Dropartifact is a deployable part of your application produced by the Build.The Continuous deployment trigger instructs Release Management to create a new release when it detects that a new artifact is available. Again, you can fine-tune the events that trigger the release, for example only processing artifacts from the master branch.

PipelineView on Visual Studio Team Services
Figure7 Pipeline View on Visual Studio Team Services

By default, the CI/CD pipeline created by the DevOps Project automatically deploys to the Devenvironment when a release is created by a continuous deployment trigger. The Dev environment requires no manual approval, Gatesare disabled, and it has only one task that deploys the sample Azure App Service.

The pipeline generated by the DevOps Project is a great starting point for demonstrations and experimentation. For production, the pipeline may require some additional love, which we’ll discuss in the next section.

Last, open the second dNCoWe4DDemo sample project and have a look at the CI/CD pipeline. Overall the pipeline looks very similar. However, drill into the build and you’ll notice that the CI build triggers a build of the container image, pushing it to the Azure Container Registry. The CD release deploys the Azure App Service on Linux, without any Linux or container expertise required.

As shown in this quick walk-through, DevOps Projects get you started quickly with a complete and functional CI/CD pipeline. The learning curve for Azure, VSTS and DevOps is significantly lower when using DevOps Projects, compared to building the solution and the CI/CD pipeline by hand. You’re now empowered to work with your stakeholders, for example security, operations and release management to take your CI/CD pipeline to the next level.

Polishing the Generated Pipeline

The CI/CD pipeline is an essential element of DevOps that helps teams consistently and continuously deliver value, at a faster pace and with lower risk. You can enhance the generated pipeline to align with your process and organizational policies. You can fully automate the testing, validation, and delivery of your software in multiple environments in production, or set up a semi-automated pipeline with approvals and gates. You may even want to raise the level of quality, security and progressive exposure of new features.

Here are a few pointers to get you started:

Greenlighting and gates: Refine your release pipeline with a set of gates in pre- and post-deployment options that integrate signals from monitoring systems and other external services (bit.ly/2E549uG).
Analyze open source projects: Continuously analyze and measure technical quality, with SonarCloud and VSTS, from your project down to each method (bit.ly/2J4v0ea).
Security validation: Continuously secure solutions within the CI/CD Pipeline. Address secure infrastructure, validate security, scan open source components for vulnerabilities, and monitor for attacks (bit.ly/2Gmczjg).
New release deployment: Use deployment rings to progressively expose a new release, and fine-tune releases in production with feature flags (red.ht/2GTsQNC).

The Importance of Package Management

If you’re thinking about ways to deliver more value more quickly to your end users, then you’re probably using packages (such as NuGet, npm and the like). Packages and DevOps have a symbiotic relationship. Package management is focused on delivering additional value through packaged products to the CI/CD pipeline and empowering build and release automation. In VSTS, the Package Management extension is designed to be a seamless part of your DevOps pipeline, making it easier to responsibly use packages from public sources, and faster to create and share packages of your own.

The easiest way to add some Package Management magic to your DevOps pipeline is with upstream sources, which connect your Package Management feed to public sources like nuget.org and npmjs.com. There are a couple big advantages to using packages from these sources through Package Management:

Every time you use a package, a saved copy is kept in your feed, which means you’re protected from issues like the left-pad incident (see bit.ly/2pO1Do5), public source outages and whatever else the world may throw at you. All you need to keep your pipeline flowing is your VSTS package feed.
In your feed, you can see all the saved copies and their provenance, so you can filter and see exactly which packages you’ve used from where.

Starting with package management takes just two steps. First, install Package Management and create a feed, as shown at bit.ly/2GDj5W9. By default, the new feed is automatically set up with upstream sources for nuget.org and npmjs.com. Then, either Visual Studio (bit.ly/2pR3aZZ) or npm (bit.ly/2J2xVUq) to use your new feed. From there, it’s business as usual. When getting started, you can run a clean build to force your packages to be saved into the feed.

Wrapping Up

This is the first of a series of articles that we’re planning to cover new and future features. We believe that transparency helps build empathy and trust, and more important, enables you to take incremental steps to adopt products and improve your process. Bookmark our timeline (bit.ly/2uv25fM), watch for future articles, and look for announcements at the upcoming Microsoft Build and Inspire events. At this point, we can mention that we’re working on a few interesting enhancements for DevOps Projects, including expanding Azure VM Resources, adding support for Go and Ruby, and extending support for databases.

Now that we’ve introduced DevOps Projects and covered the magic behind the scenes, you should feel confident exploring ways to enhance your CI/CD pipelines. Remember, DevOps isn’t a destination, it’s a journey of continuous, rapid improvement.

DevOps Resources

The arena of DevOps and CI/CD is evolving fast, and there’s a lot of ground for developers to cover. To help with the challenge, we’ve assembled a few resources and links to content that you can explore and bookmark for future reference. These resources provide the latest product updates and learnings from our DevOps transformation stories. We hope they will enable you simplify and accelerate your own DevOps transformations.

Willy-Peter Schaub is a program manager in VSTS, working at Microsoft Vancouver in beautiful British Columbia. Since the mid-80s, he’s been striving for simplicity and maintainability in software engineering. You can follow him on LinkedIn at aka.ms/willysli or on Twitter: @wpschaub.

Alex Mullans is a program manager in VSTS, making it easier and faster for teams to share and reuse code. You can follow him on Twitter: @alexmullans.

Thanks to the following technical experts for reviewing this article: ALM | DevOps Rangers, Gopinath Chigakkagari, Atul Malaviya

Discuss this article in the MSDN Magazine forum