Control access to cluster resources using Kubernetes role-based access control and Azure Active Directory identities in Azure Kubernetes Service

Azure Kubernetes Service (AKS) can be configured to use Azure Active Directory (Azure AD) for user authentication. In this configuration, you sign in to an AKS cluster using an Azure AD authentication token. Once authenticated, you can use the built-in Kubernetes role-based access control (Kubernetes RBAC) to manage access to namespaces and cluster resources based on a user's identity or group membership.

This article shows you how to:

  • Control access using Kubernetes RBAC in an AKS cluster based on Azure AD group membership.
  • Create example groups and users in Azure AD.
  • Create Roles and RoleBindings in an AKS cluster to grant the appropriate permissions to create and view resources.

Before you begin

  • This article assumes that you have an existing AKS cluster enabled with Azure AD integration. If you need an AKS cluster, see Integrate Azure AD with AKS.
  • Kubernetes RBAC is enabled by default during AKS cluster creation. If Kubernetes RBAC wasn't enabled when you originally deployed your cluster, you'll need to delete and recreate your cluster.
  • Make sure that Azure CLI version 2.0.61 or later is installed and configured. Run az --version to find the version. If you need to install or upgrade, see Install Azure CLI.
  • If using Terraform, install Terraform version 2.99.0 or later.

Use the Azure portal or Azure CLI to verify if Kubernetes RBAC is enabled.

Verify Kubernetes RBAC is enabled using the Azure portal:

  • From your browser, sign in to the Azure portal.
  • Navigate to Kubernetes services, and from the left-hand pane select Cluster configuration.
  • Under the Authentication and Authorization section, check to see if the Local accounts with Kubernetes RBAC or the Azure AD authentication with Kubernetes RBAC option is shown.

Example of Authentication and Authorization page in Azure portal.

Create demo groups in Azure AD

In this article, we'll create two user roles to show how Kubernetes RBAC and Azure AD control access to cluster resources. The following two example roles are used:

  • Application developer
    • A user named aksdev that's part of the appdev group.
  • Site reliability engineer
    • A user named akssre that's part of the opssre group.

In production environments, you can use existing users and groups within an Azure AD tenant.

  1. First, get the resource ID of your AKS cluster using the az aks show command. Then, assign the resource ID to a variable named AKS_ID so it can be referenced in other commands.

    AKS_ID=$(az aks show \
        --resource-group myResourceGroup \
        --name myAKSCluster \
        --query id -o tsv)
    
  2. Create the first example group in Azure AD for the application developers using the az ad group create command. The following example creates a group named appdev:

    APPDEV_ID=$(az ad group create --display-name appdev --mail-nickname appdev --query Id -o tsv)
    
  3. Create an Azure role assignment for the appdev group using the az role assignment create command. This assignment lets any member of the group use kubectl to interact with an AKS cluster by granting them the Azure Kubernetes Service Cluster User Role.

    az role assignment create \
      --assignee $APPDEV_ID \
      --role "Azure Kubernetes Service Cluster User Role" \
      --scope $AKS_ID
    

Tip

If you receive an error such as Principal 35bfec9328bd4d8d9b54dea6dac57b82 doesn't exist in the directory a5443dcd-cd0e-494d-a387-3039b419f0d5., wait a few seconds for the Azure AD group object ID to propagate through the directory then try the az role assignment create command again.

  1. Create a second example group for SREs named opssre.

    OPSSRE_ID=$(az ad group create --display-name opssre --mail-nickname opssre --query objectId -o tsv)
    
  2. Create an Azure role assignment to grant members of the group the Azure Kubernetes Service Cluster User Role.

    az role assignment create \
      --assignee $OPSSRE_ID \
      --role "Azure Kubernetes Service Cluster User Role" \
      --scope $AKS_ID
    

Create demo users in Azure AD

Now that we have two example groups created in Azure AD for our application developers and SREs, we'll create two example users. To test the Kubernetes RBAC integration at the end of the article, you'll sign in to the AKS cluster with these accounts.

Set the user principal name and password for application developers

Set the user principal name (UPN) and password for the application developers. The UPN must include the verified domain name of your tenant, for example aksdev@contoso.com.

The following command prompts you for the UPN and sets it to AAD_DEV_UPN so it can be used in a later command:

echo "Please enter the UPN for application developers: " && read AAD_DEV_UPN

The following command prompts you for the password and sets it to AAD_DEV_PW for use in a later command:

echo "Please enter the secure password for application developers: " && read AAD_DEV_PW

Create the user accounts

  1. Create the first user account in Azure AD using the az ad user create command. The following example creates a user with the display name AKS Dev and the UPN and secure password using the values in AAD_DEV_UPN and AAD_DEV_PW:
AKSDEV_ID=$(az ad user create \
  --display-name "AKS Dev" \
  --user-principal-name $AAD_DEV_UPN \
  --password $AAD_DEV_PW \
  --query objectId -o tsv)
  1. Add the user to the appdev group created in the previous section using the az ad group member add command.
az ad group member add --group appdev --member-id $AKSDEV_ID
  1. Set the UPN and password for SREs. The UPN must include the verified domain name of your tenant, for example akssre@contoso.com. The following command prompts you for the UPN and sets it to AAD_SRE_UPN for use in a later command:
echo "Please enter the UPN for SREs: " && read AAD_SRE_UPN
  1. The following command prompts you for the password and sets it to AAD_SRE_PW for use in a later command:
echo "Please enter the secure password for SREs: " && read AAD_SRE_PW
  1. Create a second user account. The following example creates a user with the display name AKS SRE and the UPN and secure password using the values in AAD_SRE_UPN and AAD_SRE_PW:
# Create a user for the SRE role
AKSSRE_ID=$(az ad user create \
  --display-name "AKS SRE" \
  --user-principal-name $AAD_SRE_UPN \
  --password $AAD_SRE_PW \
  --query objectId -o tsv)

# Add the user to the opssre Azure AD group
az ad group member add --group opssre --member-id $AKSSRE_ID

Create AKS cluster resources for app devs

We have our Azure AD groups, users, and Azure role assignments created. Now, we'll configure the AKS cluster to allow these different groups access to specific resources.

  1. Get the cluster admin credentials using the az aks get-credentials command. In one of the following sections, you get the regular user cluster credentials to see the Azure AD authentication flow in action.
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster --admin
  1. Create a namespace in the AKS cluster using the kubectl create namespace command. The following example creates a namespace name dev:
kubectl create namespace dev

Note

In Kubernetes, Roles define the permissions to grant, and RoleBindings apply them to desired users or groups. These assignments can be applied to a given namespace, or across the entire cluster. For more information, see Using Kubernetes RBAC authorization.

If the user you grant the Kubernetes RBAC binding for is in the same Azure AD tenant, assign permissions based on the userPrincipalName (UPN). If the user is in a different Azure AD tenant, query for and use the objectId property instead.

  1. Create a Role for the dev namespace, which grants full permissions to the namespace. In production environments, you can specify more granular permissions for different users or groups. Create a file named role-dev-namespace.yaml and paste the following YAML manifest:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dev-user-full-access
  namespace: dev
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["*"]
  verbs: ["*"]
- apiGroups: ["batch"]
  resources:
  - jobs
  - cronjobs
  verbs: ["*"]
  1. Create the Role using the kubectl apply command and specify the filename of your YAML manifest.
kubectl apply -f role-dev-namespace.yaml
  1. Get the resource ID for the appdev group using the az ad group show command. This group is set as the subject of a RoleBinding in the next step.
az ad group show --group appdev --query id -o tsv
  1. Create a RoleBinding for the appdev group to use the previously created Role for namespace access. Create a file named rolebinding-dev-namespace.yaml and paste the following YAML manifest. On the last line, replace groupObjectId with the group object ID output from the previous command.
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dev-user-access
  namespace: dev
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: dev-user-full-access
subjects:
- kind: Group
  namespace: dev
  name: groupObjectId

Tip

If you want to create the RoleBinding for a single user, specify kind: User and replace groupObjectId with the user principal name (UPN) in the above sample.

  1. Create the RoleBinding using the kubectl apply command and specify the filename of your YAML manifest:
kubectl apply -f rolebinding-dev-namespace.yaml

Create the AKS cluster resources for SREs

Now, we'll repeat the previous steps to create a namespace, Role, and RoleBinding for the SREs.

  1. Create a namespace for sre using the kubectl create namespace command.
kubectl create namespace sre
  1. Create a file named role-sre-namespace.yaml and paste the following YAML manifest:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sre-user-full-access
  namespace: sre
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["*"]
  verbs: ["*"]
- apiGroups: ["batch"]
  resources:
  - jobs
  - cronjobs
  verbs: ["*"]
  1. Create the Role using the kubectl apply command and specify the filename of your YAML manifest.
kubectl apply -f role-sre-namespace.yaml
  1. Get the resource ID for the opssre group using the az ad group show command.
az ad group show --group opssre --query id -o tsv
  1. Create a RoleBinding for the opssre group to use the previously created Role for namespace access. Create a file named rolebinding-sre-namespace.yaml and paste the following YAML manifest. On the last line, replace groupObjectId with the group object ID output from the previous command.
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sre-user-access
  namespace: sre
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: sre-user-full-access
subjects:
- kind: Group
  namespace: sre
  name: groupObjectId
  1. Create the RoleBinding using the kubectl apply command and specify the filename of your YAML manifest.
kubectl apply -f rolebinding-sre-namespace.yaml

Interact with cluster resources using Azure AD identities

Now, we'll test that the expected permissions work when you create and manage resources in an AKS cluster. In these examples, we'll schedule and view pods in the user's assigned namespace, and try to schedule and view pods outside of the assigned namespace.

  1. Reset the kubeconfig context using the az aks get-credentials command. In a previous section, you set the context using the cluster admin credentials. The admin user bypasses Azure AD sign-in prompts. Without the --admin parameter, the user context is applied that requires all requests to be authenticated using Azure AD.
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster --overwrite-existing
  1. Schedule a basic NGINX pod using the kubectl run command in the dev namespace.
kubectl run nginx-dev --image=mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine --namespace dev
  1. Enter the credentials for your own appdev@contoso.com account created at the start of the article as the sign-in prompt. Once you're successfully signed in, the account token is cached for future kubectl commands. The NGINX is successfully schedule, as shown in the following example output:
$ kubectl run nginx-dev --image=mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine --namespace dev

To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code B24ZD6FP8 to authenticate.

pod/nginx-dev created
  1. Use the kubectl get pods command to view pods in the dev namespace.
kubectl get pods --namespace dev
  1. Ensure the status of the NGINX pod is Running. The output will look similar to the following output:
$ kubectl get pods --namespace dev

NAME        READY   STATUS    RESTARTS   AGE
nginx-dev   1/1     Running   0          4m

Create and view cluster resources outside of the assigned namespace

Try to view pods outside of the dev namespace. Use the kubectl get pods command again, this time to see --all-namespaces.

kubectl get pods --all-namespaces

The user's group membership doesn't have a Kubernetes Role that allows this action, as shown in the following example output:

Error from server (Forbidden): pods is forbidden: User "aksdev@contoso.com" cannot list resource "pods" in API group "" at the cluster scope

In the same way, try to schedule a pod in a different namespace, such as the sre namespace. The user's group membership doesn't align with a Kubernetes Role and RoleBinding to grant these permissions, as shown in the following example output:

$ kubectl run nginx-dev --image=mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine --namespace sre

Error from server (Forbidden): pods is forbidden: User "aksdev@contoso.com" cannot create resource "pods" in API group "" in the namespace "sre"

Test the SRE access to the AKS cluster resources

To confirm that our Azure AD group membership and Kubernetes RBAC work correctly between different users and groups, try the previous commands when signed in as the opssre user.

  1. Reset the kubeconfig context using the az aks get-credentials command that clears the previously cached authentication token for the aksdev user.
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster --overwrite-existing
  1. Try to schedule and view pods in the assigned sre namespace. When prompted, sign in with your own opssre@contoso.com credentials created at the start of the article.
kubectl run nginx-sre --image=mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine --namespace sre
kubectl get pods --namespace sre

As shown in the following example output, you can successfully create and view the pods:

$ kubectl run nginx-sre --image=mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine --namespace sre

3. To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code BM4RHP3FD to authenticate.

pod/nginx-sre created

$ kubectl get pods --namespace sre

NAME        READY   STATUS    RESTARTS   AGE
nginx-sre   1/1     Running   0
  1. Try to view or schedule pods outside of assigned SRE namespace.
kubectl get pods --all-namespaces
kubectl run nginx-sre --image=mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine --namespace dev

These kubectl commands fail, as shown in the following example output. The user's group membership and Kubernetes Role and RoleBindings don't grant permissions to create or manager resources in other namespaces.

$ kubectl get pods --all-namespaces
Error from server (Forbidden): pods is forbidden: User "akssre@contoso.com" cannot list pods at the cluster scope

$ kubectl run nginx-sre --image=mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine --namespace dev
Error from server (Forbidden): pods is forbidden: User "akssre@contoso.com" cannot create pods in the namespace "dev"

Clean up resources

In this article, you created resources in the AKS cluster and users and groups in Azure AD. To clean up all of the resources, run the following commands:

# Get the admin kubeconfig context to delete the necessary cluster resources.

az aks get-credentials --resource-group myResourceGroup --name myAKSCluster --admin

# Delete the dev and sre namespaces. This also deletes the pods, Roles, and RoleBindings.

kubectl delete namespace dev
kubectl delete namespace sre

# Delete the Azure AD user accounts for aksdev and akssre.

az ad user delete --upn-or-object-id $AKSDEV_ID
az ad user delete --upn-or-object-id $AKSSRE_ID

# Delete the Azure AD groups for appdev and opssre. This also deletes the Azure role assignments.

az ad group delete --group appdev
az ad group delete --group opssre

Next steps