Edit

Best practices for authentication and authorization in Azure Kubernetes Service (AKS)

  • Article

As you deploy and maintain clusters in Azure Kubernetes Service (AKS), you implement ways to manage access to resources and services. Without these controls:

  • Accounts could have access to unnecessary resources and services.
  • Tracking credentials used to make changes can be difficult.

In this article, we discuss what recommended practices a cluster operator can follow to manage access and identity for AKS clusters. You'll learn how to:

  • Authenticate AKS cluster users with Azure Active Directory (Azure AD).
  • Control access to resources with Kubernetes role-based access control (Kubernetes RBAC).
  • Use Azure RBAC to granularly control access to the AKS resource, the Kubernetes API at scale, and the kubeconfig.
  • Use a managed identity to authenticate pods with other services.

Use Azure Active Directory (Azure AD)

Best practice guidance

Deploy AKS clusters with Azure AD integration. Using Azure AD centralizes the identity management layer. Any change in user account or group status is automatically updated in access to the AKS cluster. Scope users or groups to the minimum permissions amount using Roles, ClusterRoles, or Bindings.

Your Kubernetes cluster developers and application owners need access to different resources. Kubernetes lacks an identity management solution for you to control the resources with which users can interact. Instead, you can integrate your cluster with an existing identity solution like Azure AD, an enterprise-ready identity management solution.

With Azure AD-integrated clusters in AKS, you create Roles or ClusterRoles defining access permissions to resources. You then bind the roles to users or groups from Azure AD. Learn more about these Kubernetes RBAC in the next section. Azure AD integration and how you control access to resources can be seen in the following diagram:

Cluster-level authentication for Azure Active Directory integration with AKS

  1. Developer authenticates with Azure AD.
  2. The Azure AD token issuance endpoint issues the access token.
  3. The developer performs an action using the Azure AD token, such as kubectl create pod.
  4. Kubernetes validates the token with Azure AD and fetches the developer's group memberships.
  5. Kubernetes RBAC and cluster policies are applied.
  6. The developer's request is successful based on previous validation of Azure AD group membership and Kubernetes RBAC and policies.

To create an AKS cluster that uses Azure AD, see Integrate Azure Active Directory with AKS.

Use Kubernetes role-based access control (Kubernetes RBAC)

Best practice guidance

Define user or group permissions to cluster resources with Kubernetes RBAC. Create roles and bindings that assign the least amount of permissions required. Integrate with Azure AD to automatically update any user status or group membership change and keep access to cluster resources current.

In Kubernetes, you provide granular access control to cluster resources. You define permissions at the cluster level, or to specific namespaces. You determine what resources can be managed and with what permissions. You then apply these roles to users or groups with a binding. For more information about Roles, ClusterRoles, and Bindings, see Access and identity options for Azure Kubernetes Service (AKS).

For example, you create a role with full access to resources in the namespace named finance-app, as shown in the following example YAML manifest:

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: finance-app-full-access-role
  namespace: finance-app
rules:
- apiGroups: [""]
  resources: ["*"]
  verbs: ["*"]

You then create a RoleBinding and bind the Azure AD user developer1@contoso.com to it, as shown in the following YAML manifest:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: finance-app-full-access-role-binding
  namespace: finance-app
subjects:
- kind: User
  name: developer1@contoso.com
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: finance-app-full-access-role
  apiGroup: rbac.authorization.k8s.io

When developer1@contoso.com is authenticated against the AKS cluster, they have full permissions to resources in the finance-app namespace. In this way, you logically separate and control access to resources. Use Kubernetes RBAC with Azure AD-integration.

To learn how to use Azure AD groups to control access to Kubernetes resources using Kubernetes RBAC, see Control access to cluster resources using role-based access control and Azure Active Directory identities in AKS.

Use Azure RBAC

Best practice guidance

Use Azure RBAC to define the minimum required user and group permissions to AKS resources in one or more subscriptions.

There are two levels of access needed to fully operate an AKS cluster:

  • Access the AKS resource on your Azure subscription.

    This access level allows you to:

    • Control scaling or upgrading your cluster using the AKS APIs
    • Pull your kubeconfig.

    To learn how to control access to the AKS resource and the kubeconfig, see Limit access to cluster configuration file.

  • Access to the Kubernetes API.

    This access level is controlled either by:

    • Kubernetes RBAC (traditionally) or
    • By integrating Azure RBAC with AKS for kubernetes authorization.

    To learn how to granularly grant permissions to the Kubernetes API using Azure RBAC, see Use Azure RBAC for Kubernetes authorization.

Use pod-managed identities

Don't use fixed credentials within pods or container images, as they are at risk of exposure or abuse. Instead, use pod identities to automatically request access using Azure AD.

Note

Pod identities are intended for use with Linux pods and container images only. Pod-managed identities (preview) support for Windows containers is coming soon.

To access other Azure resources, like Azure Cosmos DB, Key Vault, or Blob storage, the pod needs authentication credentials. You could define authentication credentials with the container image or inject them as a Kubernetes secret. Either way, you would need to manually create and assign them. Usually, these credentials are reused across pods and aren't regularly rotated.

With pod-managed identities (preview) for Azure resources, you automatically request access to services through Azure AD. Pod-managed identities is currently in preview for AKS. Refer to the Use Azure Active Directory pod-managed identities in Azure Kubernetes Service (Preview) documentation to get started.

Note

If you have enabled Azure AD pod-managed identity on your AKS cluster or are considering implementing it, we recommend you first review the workload identity overview article to understand our recommendations and options to set up your cluster to use an Azure AD workload identity (preview). This authentication method replaces pod-managed identity (preview), which integrates with the Kubernetes native capabilities to federate with any external identity providers.

The open source Azure AD pod-managed identity (preview) in Azure Kubernetes Service has been deprecated as of 10/24/2022.

Azure Active Directory pod-managed identity (preview) supports two modes of operation:

  • Standard mode: In this mode, the following 2 components are deployed to the AKS cluster:

    • Managed Identity Controller(MIC): A Kubernetes controller that watches for changes to pods, AzureIdentity and AzureIdentityBinding through the Kubernetes API Server. When it detects a relevant change, the MIC adds or deletes AzureAssignedIdentity as needed. Specifically, when a pod is scheduled, the MIC assigns the managed identity on Azure to the underlying virtual machine scale set used by the node pool during the creation phase. When all pods using the identity are deleted, it removes the identity from the virtual machine scale set of the node pool, unless the same managed identity is used by other pods. The MIC takes similar actions when AzureIdentity or AzureIdentityBinding are created or deleted.

    • Node Managed Identity (NMI): is a pod that runs as a DaemonSet on each node in the AKS cluster. NMI intercepts security token requests to the Azure Instance Metadata Service on each node. It redirects requests to itself and validates if the pod has access to the identity it's requesting a token for, and fetch the token from the Azure Active Directory tenant on behalf of the application.

  • Managed mode: In this mode, there's only NMI. The identity needs to be manually assigned and managed by the user. For more information, see Pod Identity in Managed Mode. In this mode, when you use the az aks pod-identity add command to add a pod identity to an Azure Kubernetes Service (AKS) cluster, it creates the AzureIdentity and AzureIdentityBinding in the namespace specified by the --namespace parameter, while the AKS resource provider assigns the managed identity specified by the --identity-resource-id parameter to virtual machine scale set of each node pool in the AKS cluster.

Note

If you instead decide to install the Azure Active Directory pod-managed identity using the AKS cluster add-on, setup uses the managed mode.

The managed mode provides the following advantages over the standard:

  • Identity assignment on the virtual machine scale set of a node pool can take up 40-60s. With cronjobs or applications that require access to the identity and can't tolerate the assignment delay, it's best to use managed mode as the identity is pre-assigned to the virtual machine scale set of the node pool. Either manually or using the az aks pod-identity add command.
  • In standard mode, MIC requires write permissions on the virtual machine scale set used by the AKS cluster and Managed Identity Operator permission on the user-assigned managed identities. When running in managed mode, since there's no MIC, the role assignments aren't required.

Instead of manually defining credentials for pods, pod-managed identities request an access token in real time, using it to access only their assigned resources. In AKS, there are two components that handle the operations to allow pods to use managed identities:

  • The Node Management Identity (NMI) server is a pod that runs as a DaemonSet on each node in the AKS cluster. The NMI server listens for pod requests to Azure services.
  • The Azure Resource Provider queries the Kubernetes API server and checks for an Azure identity mapping that corresponds to a pod.

When pods request a security token from Azure Active Directory to access to an Azure resource, network rules redirect the traffic to the NMI server.

  1. The NMI server:

    • Identifies pods requesting access to Azure resources based on their remote address.
    • Queries the Azure Resource Provider.

  2. The Azure Resource Provider checks for Azure identity mappings in the AKS cluster.

  3. The NMI server requests an access token from Azure AD based on the pod's identity mapping.

  4. Azure AD provides access to the NMI server, which is returned to the pod.

    • This access token can be used by the pod to then request access to resources in Azure.

In the following example, a developer creates a pod that uses a managed identity to request access to Azure SQL Database:

Pod identities allow a pod to automatically request access to other resources.

  1. Cluster operator creates a service account to map identities when pods request access to resources.
  2. The NMI server is deployed to relay any pod requests, along with the Azure Resource Provider, for access tokens to Azure AD.
  3. A developer deploys a pod with a managed identity that requests an access token through the NMI server.
  4. The token is returned to the pod and used to access Azure SQL Database

To use Pod-managed identities, see Use Azure Active Directory pod-managed identities in Azure Kubernetes Service (preview).

Next steps

This best practices article focused on authentication and authorization for your cluster and resources. To implement some of these best practices, see the following articles:

For more information about cluster operations in AKS, see the following best practices: