Automation for Azure Arc-enabled Kubernetes

Azure Arc-enabled Kubernetes clusters allow you to manage Kubernetes clusters that are hosted outside Azure, on your corporate network, or on another cloud provider. This article provides an overview of automation for cluster onboarding and adding extra capabilities through cluster extensions. The following sections contain considerations and recommendations for your operations team(s) to use when onboarding and automating Azure Arc-enabled clusters throughout their lifecycles.

Architecture

The following diagram contains a conceptual reference architecture that highlights onboarding and automation design areas for Azure Arc-enabled Kubernetes:

Design Considerations

Consider the following before you onboard Azure Arc-enabled Kubernetes clusters to Azure:

Requirements

• Review the list of validated distributions for the supported Kubernetes distribution your cluster runs.
• Review the Azure Arc-enabled Kubernetes agent overview.

Environment Preparation

• You must have a cluster admin role on your Kubernetes cluster before you can deploy and configure the Azure Arc-enabled Kubernetes agent.

Onboard Azure Arc-enabled Kubernetes

• Plan to install and configure the Azure Arc-enabled Kubernetes agent on your cluster. You should usually deploy this agent using your organization's standard automation tool(s).

Cluster Extensions

• Determine which Azure capabilities you want on your Azure Arc-enabled Kubernetes cluster. Some services require a cluster extension to be deployed.
  • For more information about extensions, see the Extensions management critical design area.

Network Connectivity

• Your cluster has connectivity from your on-premises network or third-party cloud providers to Azure, directly connected via a proxy server or private endpoint.
  • For specific design considerations and recommendations, see the Network connectivity for Azure Arc-enabled Kubernetes critical design area.

Agent lifecycle automation

• Create a strategy to update Azure Arc agents and Azure Arc-enabled Kubernetes extensions.

Design recommendations

The following sections contain design recommendations for Azure Arc-enabled Kubernetes clusters.

Environment preparation

• Review the requirements for onboarding Kubernetes clusters onto Azure Arc.
• Review the Identity and access management critical design area for required permissions.

Onboard Azure Arc-enabled Kubernetes clusters

• When onboarding multiple clusters, create a service principal and onboard your clusters using tooling like Azure DevOps or GitHub Actions to manage Kubernetes clusters.

Arc-enabled Kubernetes extensions

• If you're deploying an extension only to a specific Azure Arc-enabled Kubernetes cluster or clusters, automate the installation of these extensions through Azure CLI and/or ARM templates using tools such as Azure DevOps or GitHub Actions.
• If an extension is common across all your Arc-enabled Kubernetes clusters or large groups of Arc-enabled Kubernetes clusters, use Azure Policy to automate the deployment of Arc extensions at scale.
  • Review the Extensions Management critical design area. The following is an overview of its steps:
    • Create an initiative to deploy Azure Arc-enabled Kubernetes extensions at scale.
    • Use a "DeployIfNotExists" policy effect to ensure your Azure Arc-enabled Kubernetes extensions are deployed automatically. As you onboard more Kubernetes clusters, use Azure Policy to remediate any clusters where the extensions have been removed.
  • For more details on using Azure Policy with Azure Arc-enabled Kubernetes clusters, review the Governance and security disciplines critical design area.

Agent and extensions lifecycle automation

During the onboarding process, Azure Arc-enabled Kubernetes provisions agents into your Kubernetes cluster. Agent versions change as Azure Arc technologies evolve, so update your agents frequently.

• Enable the auto-upgrade feature for Azure Arc agents running inside your cluster, which is the default behavior when onboarding a cluster to Azure Arc.

  • For more information on the auto-upgrade feature and version support policy, review the Upgrade Agents guidance and the extensions management design area.

• Extensions also require updates in your cluster. For any extension installed on your cluster, we recommend leaving automatic minor version upgrades enabled during provisioning (the default). For major version upgrades, documentation provides a migration path to the extension major release.

  • For more information, review the Extensions management critical design area.

Next steps

For more information about your hybrid and multicloud journey, see the following articles.

• Review the prerequisites for Azure Arc-enabled Kubernetes.
• Review the validated Kubernetes distributions for Azure Arc-enabled Kubernetes.
• Learn about GitHub Actions for Azure to understand how you can use GitHub Actions to automate Azure resource deployments.
• Learn about Azure Pipelines to understand how you can use Azure Pipelines for automation.
• Experience Azure Arc-enabled Kubernetes automated scenarios with Azure Arc Jumpstart.
• Learn about Azure Arc via the Azure Arc learning path.
• See Frequently Asked Questions - Azure Arc-enabled to find answers to most common questions.