Automate API deployments with APIOps

API Management
Azure DevOps
Pipelines

APIOps applies GitOps and DevOps to API deployments. It uses version control to manage APIs and creates an audit trail of changes to APIs, policies, and operations. With APIOps, API developers can review and audit APIs earlier and more frequently, catching and resolving deviations from API standards faster for better specifications and API quality. The more APIs that you build and deploy with an APIOps approach, the greater the consistency between APIs.

This APIOps architecture uses Azure API Management as the API management platform. Azure DevOps organizes API management. Azure Repos provides Git functionality and Azure Pipelines creates the CI/CD pipeline.

Potential use cases

  • Any organization developing and managing APIs
  • Highly regulated industries: insurance, banking, finance, government.

Architecture

Diagram of the architecture for automated API deployments using APIOps on Azure.

Download a Visio file of this architecture.

Workflow

  1. API operators run the extractor pipeline to synchronize the Git repository with the API Management instance and populate the Git repository with API Management objects in the required format.

  2. If a developer changes an API in the API Management instance, it creates a pull request (PR) for operators to review and merge the changes in the Git repository.

  3. API developers clone the Git repository, create a branch, and create API definitions by using the OpenAPI specification or tools of their choice.

  4. API developers push the changes to the repository and create a PR for review.

  5. The PR can be automatically approved or reviewed, depending on the level of control that's required.

  6. After approving and merging the changes, the publishing pipeline deploys the latest changes to the API Management instance.

  7. API operators create and modify API Management policies, diagnostics, products, and other relevant objects, and then commit the changes.

  8. The changes are reviewed, and they're merged after approval.

  9. After merging the changes, the publishing pipeline deploys the changes by using the API-definitions process.

Components

  • Azure API Management creates consistent, modern API gateways for back-end services. Besides routing API calls to back ends, this platform also verifies credentials, enforces usage quotas, and logs metadata.

  • Azure DevOps is a service for managing your development lifecycle end-to-end, including planning and project management, code management, and continuing to build and release.

  • Azure Pipelines enables continuous integration (CI) and continuous delivery (CD) to test and build your code and ship it to any target.

  • Azure Repos is a set of version control tools, including standard Git, that you can use to manage your code.

Alternatives

The original implementation of this solution used a modified version of the Azure API Management DevOps Resource Kit. We extended the utilities, such as the extractor and the creator, to deploy changes to the API. The implementation is now more technology agnostic. You can now use Terraform, Azure Resource Manager, PowerShell, the REST API, and so on, to easily format and push newly extracted changes back to the portal.

This solution uses Azure Repos to provide Git functionality and Azure Pipelines provides the pipelines. You can use any comparable technologies.

Considerations

These considerations implement the pillars of the Azure Well-Architected Framework (WAF). WAF gives you guiding tenets to improve the quality of a workload. For more information, see Microsoft Azure Well-Architected Framework.

Performance efficiency

APIOps has many benefits, but as API Management landscapes grow, so does the complexity of managing them. This solution helps meet challenges like:

  • Keeping an overview of all environments and API Management instances.
  • Tracking critical changes to APIs and policies.
  • Creating an audit trail for all deployed changes.

Security

This solution provides several security-related benefits. Individual developers—and even operators—don't directly access the API Management instance to apply changes or updates. Instead, users push changes to a Git repository, and the extractor and publishing pipelines read and apply them to the API Management instance. This approach follows the security best practice of least privilege by not giving teams write permissions to the API Management service instance. In diagnostic or troubleshooting scenarios, you can grant elevated permissions for a limited time on a case-by-case basis.

To make sure the API Management instances are using best practices for security, you can extend this solution to enforce API best practices by using third-party tools and unit testing. Teams can provide early feedback via PR review if the proposed changes to an API or policy violate standards.

Apart from the task of setting up repository permissions, consider implementing the following security measures in Git repositories that synchronize to API Management instances:

  • Pull Request (PR) Review: Use branches and protect the branches that represent the state of the API Management instances from having changes pushed to them directly. Require PRs to have at least one reviewer to enforce the four-eyes principle.
  • Immutable history: Only allow new commits on top of existing changes. Immutable history is especially important for auditing purposes.
  • Multi-factor authentication: Require your users to activate two-factor authentication.
  • Signed Commits: Allow only signed commits that can't be altered after the fact.

Cost optimization

Cost optimization is about looking at ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Overview of the cost optimization pillar.

  • Use the Azure pricing calculator to estimate costs.

  • API Management offers the following tiers: Consumption, Developer, Basic, Standard, and Premium.

  • GitHub offers a free service. However, to use advanced security-related features, such as code owners or required reviewers, you need the Team plan. For more information, see GitHub pricing.

Operational excellence

APIOps can increase DevOps productivity for API development and deployments. One of the most useful features is the ability to use Git operations to quickly roll back changes that behave unexpectedly. The commit graph contains all commits, so it can help with the post-mortem analysis.

API operators often manage multiple environments for the same set of APIs. It's typical to have several stages of an API deployed to different API Management instances or in a shared API Management instance. The Git repository, which is the single source of truth, shows which versions of applications are currently deployed to a cluster.

When someone makes a PR in the Git repository, the API operator knows they have new code to review. For example, when a developer takes the OpenAPI specification and builds the API implementation, they add this new code to the repository. The operators can review the PR and make sure that the API that's been submitted for review meets best practices and standards.

Deploy this scenario

Deploying this solution involves these steps:

  • Develop the API in the portal or make changes to the OpenAPI specification by using a tool of your choice.

    • If you make changes in the portal, you can run the extractor to automatically extract all the APIs and other relevant policies, operations, and configurations from API Management. You can synchronize this information to the git repository.

    • Optionally, use the Azure DevOps CLI to create a new pull request.

  • The extractor workflow includes the following steps that you take:

    • Run a pipeline that downloads changes in the portal to the API Management instance.

    • Enter the names of the branch, the API Management instance, and the resource group.

      Screenshot of 'Run pipeline', where you enter the names of the API Management instance and the resource group.

  • In our scenario, the pipeline that downloads changes in the portal to the API Management instance has the following stages: Build extractor, Create artifacts from portal, and Create template branch.

    • Build extractor

      This stage builds the extractor code.

    • Create artifacts from portal

      This stage runs the extractor and creates artifacts that resemble a Git repository structure like that shown in the following screenshot:

      Screenshot of 'APIM-automation' that shows 'apim-instances' and a folder hierarchy.

      • Create template branch

        After generating the artifact, this stage creates a PR with the changes extracted for the platform team to review.

        The first time you run the extractor, it pulls everything from the Git repository. The PR that's created will have all the APIs, policies, artifacts, and so on.

        Later extractions have only changes made before the extraction in the PR. Sometimes changes might be only to the specification of an API, which is the case in the following example of a PR.

        Screenshot of an example pull request after an extraction that shows proposed changes to a file named 'specification.yml'.

  • A reviewer goes to Pull Requests to view the updated pull requests. You can also configure automatic approvals to automate this step.

    Screenshot of an example pull request that shows changes to content in 'policy.xml' and changes only to whitespace in other files.

  • After approving the PR, it triggers another pipeline that publishes from API Management to the portal. In our example, it has the following stages: build creator, build terminator, and publish APIM instances.

    Screenshot of the stages in APIM-publish-to-portal, a pipeline.

    • The build creator stage handles creation of new APIs.
    • The build terminator stage handles any deletions.
    • The publish APIM instances stage publishes changes to the API Management instance.

    Screenshot that shows the jobs in an example run of APIM-publish-to-portal, a pipeline.

    After this pipeline runs successfully, it publishes the changes in the API Management instance.

Contributors

This article is being updated and maintained by Microsoft. It was originally written by the following contributors.

Principal author:

To see non-public LinkedIn profiles, sign into LinkedIn.

Next steps