Edit

Share via

Access a private Azure Kubernetes Service (AKS) cluster using the command invoke or Run command feature

When you access a private AKS cluster, you need to connect to the cluster from the cluster virtual network, a peered network, or a configured private endpoint. These approaches require configuring a VPN, Express Route, deploying a jumpbox within the cluster virtual network, or creating a private endpoint inside of another virtual network.

With the Azure CLI, you can use command invoke to access private clusters without the need to configure a VPN or Express Route. command invoke allows you to remotely invoke commands, like kubectl and helm, on your private cluster through the Azure API without directly connecting to the cluster. The Microsoft.ContainerService/managedClusters/runcommand/action and Microsoft.ContainerService/managedclusters/commandResults/read actions control the permissions for using command invoke.

With the Azure portal, you can use the Run command feature to run commands on your private cluster. The Run command feature uses the same command invoke functionality to run commands on your cluster.

Tip

You can use Microsoft Copilot in Azure to run kubectl commands in the Azure portal. For more information, see Work with AKS clusters efficiently using Microsoft Copilot in Azure.

The pod created by the Run command provides kubectl and helm for operating your cluster. jq, xargs, grep, and awk are available for Bash support.

Before you begin

Before you begin, make sure you have the following resources and permissions:

  • An existing private cluster. If you don't have one, see Create a private AKS cluster.
  • The Azure CLI version 2.24.0 or later. Run az --version to find the version. If you need to install or upgrade, see Install Azure CLI.
  • Access to the Microsoft.ContainerService/managedClusters/runcommand/action and Microsoft.ContainerService/managedclusters/commandResults/read roles on the cluster.

Limitations

This feature is designed to simplify cluster access and is not designed for programmatic access. You should leverage direct API access via Bastion, VPN, or Express Route for programmatic calls to your cluster.

If you have a program invoke Kubernetes using Run command, the following disadvantages apply:

  • You only get exitCode and text output, and you lose API level details.
  • One extra hop introduces extra failure points.

The pod created by the Run command has 200m CPU and 500Mi memory requests, and a 500m CPU and 1Gi memory limits. In cases where all your node is full or over committed, the pod might be unable to be scheduled within the ARM API timeout of 60 seconds. This means that the Run command invocation would fail.

command invoke runs the commands from your cluster, so any commands ran in this manner are subject to your configured networking restrictions and any other configured restrictions. Make sure there are enough nodes and resources in your cluster to schedule this command pod.

Note

The output for command invoke is limited to 512kB in size.

Run commands on your AKS cluster

Below are examples of how to use az aks command invoke to execute commands against a private AKS cluster. These examples assume you have an existing resource group and AKS cluster.

Use command invoke to run a single command

You can run a command on your cluster using the az aks command invoke --command command. The following example command runs the kubectl get pods -n kube-system command on the myPrivateCluster cluster in myResourceGroup.

First, set environment variables for your resource group and cluster name to use in subsequent commands.

export AKS_RESOURCE_GROUP="myResourceGroup"
export AKS_CLUSTER_NAME="myPrivateCluster"

The environment variables above will allow you to run AKS commands in the next sections without having to rewrite their names.

To run a single kubectl command on your AKS cluster:

az aks command invoke \
  --resource-group $AKS_RESOURCE_GROUP \
  --name $AKS_CLUSTER_NAME \
  --command "kubectl get pods -n kube-system"

Use command invoke to run multiple commands

You can also run multiple commands. The following example executes three helm commands in sequence on the cluster.

az aks command invoke \
  --resource-group $AKS_RESOURCE_GROUP \
  --name $AKS_CLUSTER_NAME \
  --command "helm repo add bitnami https://charts.bitnami.com/bitnami && helm repo update && helm install my-release bitnami/nginx"

Use command invoke to run commands with an attached file

When using the --file parameter with az aks command invoke, the file must exist and be accessible in your current working directory. Below, we create a minimal deployment file for demonstration.

To run a command with a file attached, first create a Kubernetes manifest file named deployment.yaml. The following example creates a small nginx deployment and applies it with command invoke:

cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-demo
  template:
    metadata:
      labels:
        app: nginx-demo
    spec:
      containers:
      - name: nginx
        image: nginx:1.21.6
        ports:
        - containerPort: 80
EOF

az aks command invoke \
  --resource-group $AKS_RESOURCE_GROUP \
  --name $AKS_CLUSTER_NAME \
  --command "kubectl apply -f deployment.yaml -n default" \
  --file deployment.yaml

Use command invoke to run commands with all files in the current directory attached

Use only small, necessary files to avoid exceeding system size limits. Below, two minimal YAML files are created before attaching them.

cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-demo
  template:
    metadata:
      labels:
        app: nginx-demo
    spec:
      containers:
      - name: nginx
        image: nginx:1.21.6
        ports:
        - containerPort: 80
EOF

cat <<EOF > configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
data:
  welcome-message: "Hello from configmap"
EOF

az aks command invoke \
  --resource-group $AKS_RESOURCE_GROUP \
  --name $AKS_CLUSTER_NAME \
  --command "kubectl apply -f deployment.yaml -f configmap.yaml -n default" \
  --file deployment.yaml \
  --file configmap.yaml

Disable Run command

You can disable the Run command feature by setting .properties.apiServerAccessProfile.disableRunCommand to true.

Troubleshooting

For information on the most common issues with az aks command invoke and how to fix them, see Resolve az aks command invoke failures.

Next steps

In this article, you learned how to access a private cluster and run commands on that cluster. For more information on AKS clusters, see the following articles: