Edit

Run containers in a hybrid environment

Container Registry
Arc
Container Instances
Azure Cosmos DB
Kubernetes Service
Monitor

This reference architecture illustrates how developers can create, manage, and monitor deployed containers in the public cloud, across multiple clouds, and on-premises.

Architecture

The diagram illustrates a developer team that deploys its container images to a Microsoft Azure Container Registry. Subsequently, the container images are pulled and deployed to either an on-premises or cloud-based Kubernetes cluster. The containers are monitored using Azure Monitor and the container images are scanned and monitored using Azure Container Registry.

Download a Visio file of this architecture.

Components

  • Azure Container Registry is a service that creates a managed registry. It builds, stores, and manages container images and can store containerized machine learning models.
  • Azure Kubernetes Service (AKS) is a managed service that offers a managed Kubernetes cluster with elastic scale-out functionality. In this architecture, a local Kubernetes cluster is used to run multiple containers on-premises.
  • Azure Container Instances runs containers on-demand in a serverless Azure environment. Azure Container Instances is a low-friction method of running containers that doesn't require a full Docker host or Kubernetes installation.
  • Azure Cosmos DB is a multiple model database that can serve data elastically at a massive scale. It was designed for applications that are globally distributed in a multi-write model.
  • Azure Key Vault is a hardware-backed credential management service that has tight integration with Microsoft identity services and compute resources.
  • Azure Policy enforces standards and assesses compliance for targeted resources that are deployed to Azure.
  • Azure Private Link creates a private endpoint in your virtual network that you can use to communicate with Azure platform as a service (PaaS) without exposing your service to the public internet.
  • Azure Monitor is an all-encompassing suite of monitoring services for applications that deploy in Azure or on-premises.
  • Microsoft Defender for Cloud is a unified security management and threat protection system for workloads across on-premises, multiple clouds, and Azure.

Scenario details

Potential use cases

Typical uses for this architecture include:

  • Web applications that have internal and external components that deploy both to the public cloud and on-premises by using shared container images.
  • Modern deployment testing cycles with quality analysis, testing, development, or staging that's hosted on-premises and in the public cloud.

Recommendations

Azure Container Registry

Azure Container Registry is an enterprise container registry that can implement common best practices by protecting images from unauthorized access, replicating images across multiple geographies, preventing unnecessary ingress/egress, and optimizing costs. It supports turnkey geo-replication across multiple Azure regions, which helps you minimize latency between Azure Container Registry, your container hosts, and your development team.

Azure Container Registry includes a suite of tasks, referred to as ACR Tasks, that can manage cloud-based container image building and maintenance across a variety of operating systems. ACR Tasks can be triggered manually, by a change to source control, by a change to the base container image, or on a fixed schedule. The following are scenarios in which you could use ACR Tasks:

  • An Internet of Things (IoT) developer is building container images to run on ARM-based IoT devices. The developer might be using a Linux or macOS operating system to develop the software, but needs to perform the build on an ARM platform.
  • A software as a service (SaaS) development team builds software on Windows computers that run container images on Linux hosts. The team wants its builds to be done on a Linux host.
  • An open source project maintainer is building a container image that augments a well-known operating system base image. The maintainer wants the container image to update every time that the base image updates.

Note

ACR Tasks can standardize the build environment and perform continuous integration of your container images.

Azure Container Instances

Azure Container Instances is a low-friction, serverless compute environment for containerized applications. It's an excellent choice for container deployment because of its low management overhead and quick startup times. Container images that are stored in Azure Container Registry can deploy directly to Azure Container Instances container groups.

In this architecture, Azure Container Instances container groups are used as virtual nodes for an Azure Kubernetes Service cluster. AKS uses virtual nodes to register a virtual pod with unlimited capacity and the ability to dispatch pods by using Azure Container Instances container groups. This is ideal when you want fast provisioning of individual pods and only want to pay for the execution time per second.

Considerations

These considerations implement the pillars of the Azure Well-Architected Framework, which is a set of guiding tenets that can be used to improve the quality of a workload. For more information, see Microsoft Azure Well-Architected Framework.

Reliability

Reliability ensures your application can meet the commitments you make to your customers. For more information, see Overview of the reliability pillar.

  • Modern applications typically include a website, one or more HTTP APIs, and a connection to a data store. Applications within a container image should be stateless for maximum horizontal scale and availability. Data should be stored in a separate service that has similar availability. For guidance on designing an application that can scale to thousands of nodes, see the performance efficiency section of the Azure Well-Architected Framework.
  • AKS has a reference architecture baseline that defines each of the Well-Architected Framework categories and recommends an implementation that's in line with the category.
  • To reduce the impact of large pulls of container images, deploy Azure Container Registry in a region that's closest to the development team and to the production compute services. Consider a geo-replicated Azure Container Registry deployment for distributed teams and distributed production containers.
  • Azure Cosmos DB is a database service that supports turnkey global distribution and supports automatic failover across multiple regions. Azure Cosmos DB also has the ability to enable multiple region writes and dynamically add or remove regions.

Security

Security provides assurances against deliberate attacks and the abuse of your valuable data and systems. For more information, see Overview of the security pillar.

Use Azure Private Link to communicate to and across services in your virtual network. By doing this, you route traffic through specific subnets to reach the individual Azure services directly and protect your data from inadvertent exposure to the public internet.

Cost optimization

Cost optimization is about looking at ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Overview of the cost optimization pillar.

  • Use the Azure pricing calculator to estimate costs.
  • If your development team and production instances are in a single region, consider placing the Container Registry resource in the same region. By doing this, you minimize container push and pull latency and avoid the higher costs of the Premium Azure Container Registry service tier.
  • Configuring Azure Container Registry to use an Azure Virtual Network through an Azure Private Link service endpoint requires that the Azure Container Registry instance is deployed in the Premium tier.
  • AKS offers free cluster management. You're only billed for the compute, storage, and networking resources that AKS uses to host nodes. See Azure Virtual Machine or Azure Container Instances pricing to review pricing for each compute service.
  • If you require a specific uptime service-level agreement (SLA), you can enable the uptime SLA optional feature of AKS.
  • Azure Container Instances resources are billed by the second, based on an allocation of virtual CPU and memory resources to the container group. Allocating unnecessary compute resources can significantly increase the costs of running this architecture solution. Cost monitoring and optimization is a continuous process that should be conducted at regular intervals throughout the lifetime of your deployment. For more information on minimizing Azure Container Instances operational costs, see the cost optimization section of the Azure Well-Architected Framework.

Operational Excellence

Operational excellence covers the operations processes that deploy an application and keep it running in production. For more information, see Overview of the operational excellence pillar.

Manageability

DevOps

Performance efficiency

Performance efficiency is the ability of your workload to scale to meet the demands placed on it by users in an efficient manner. For more information, see Performance efficiency pillar overview.

Contributors

This article is maintained by Microsoft. It was originally written by the following contributors.

Principal author:

To see non-public LinkedIn profiles, sign in to LinkedIn.

Next steps

Related hybrid guidance:

Related architectures: