Resolve "az aks command invoke" failures
This article describes how to resolve az aks command invoke failures in Microsoft Azure CLI so that you can successfully connect to any Azure Kubernetes Service (AKS) cluster, especially to a private AKS cluster.
Other connection methods need to use extra configuration components, as shown in the following table.
|Connection methods||Extra configuration component|
|Virtual network||Virtual private network (VPN)|
|Peered network||Azure ExpressRoute|
az aks command invoke Azure CLI command is an alternative way of connecting to a cluster that doesn't require extra configuration components.
When you run the
az aks command invoke command, Azure CLI automatically creates a
command-<ID> pod in the
aks-command namespace to access the AKS cluster and retrieve the required information.
The following table lists common
az aks command invoke error messages. Each error message has a link to the section that describes why the error is occurring, and how to fix it.
|Operation returned an invalid status 'Not Found'||Cause 1: The pod can't be created because of node or resource constraints|
|Failed to run command in managed cluster due to kubernetes failure. details: admission webhook "validation.gatekeeper.sh" denied the request: <policy-specific-message>||Cause 2: Azure Policy doesn’t allow the pod creation|
|Error from server (Forbidden): namespaces is forbidden: User "<ID>" cannot list resource "<resource>" in API group "" at the cluster scope||Cause 3: Required roles aren't granted|
|Failed to connect to MSI. Please make sure MSI is configured correctly.
Get Token request returned: Response ;
|Cause 4: There's a Cloud Shell issue|
Cause 1: The pod can't be created because of node or resource constraints
The operation returns a
Not Found status because the
command-<ID> pod can't reach a successful state, such as
Running. (In many cases, the pod stays in the
Pending state.) In this case, the nodes aren't able to schedule the pod. This scenario can have different causes, such as the following causes:
- Resource constraints
- Nodes that have a
- Nodes that have taints that the pod can't tolerate
- Other causes
Solution 1: Change the configuration so that you can schedule and run the pod
Make sure that the
command-<ID> pod can be scheduled and run.
Cause 2: Azure Policy doesn’t allow the pod creation
If you have specific Azure policies, the
az aks command invoke command can fail because of a disallowed configuration in the
command-<ID> pod. For example, you might have an Azure policy that requires a read-only root file system or other specific configuration.
Solution 2: Exempt the namespace for policies that prohibit pod creation
We recommend that you exempt the
aks-command namespace for the associated Azure policies that don't allow the pod creation. For more information about exemption, see Understand scope in Azure Policy
To exempt an Azure Policy:
In the Azure portal, search for and select Policy.
In the Policy navigation pane, locate the Authoring section, and then select Assignments.
In the table of assignments, find the row that contains the Assignment name that you want to change, and then select the name of the assignment.
In the policy assignment page for that assignment, select Edit assignment.
Select the Parameters tab.
Clear the Only show parameters that need input or review option.
In the Namespace exclusions box, add the aks-command namespace to the list of namespaces to be excluded.
Alternatively, if the policy isn't a built-in policy, you can check the configuration of the
command-<ID> pod, and adjust the policy as necessary. To explore the pod's YAML configuration, run the following command:
kubectl get pods command-<ID> --namespace aks-command --output yaml
Cause 3: Required roles aren't granted
To use the
az aks command invoke command, you must have access to the following roles on the cluster:
If you don't have these roles, the
az aks command invoke command can't retrieve the required information.
Solution 3: Add the required roles
Cause 4: There's a Cloud Shell issue
az aks command invoke command isn't processed as expected when it's run directly in the Azure Cloud Shell environment. This is a known issue in Cloud Shell.
Solution 4a: Run the az login command first
In Cloud Shell, run the az login command before you run
az aks command invoke.
Solution 4b: Run the command on a local computer or a virtual machine
az aks command invoke command on a local computer or any virtual machine (VM) that has Azure CLI installed.
Third-party information disclaimer
The third-party products that this article discusses are manufactured by companies that are independent of Microsoft. Microsoft makes no warranty, implied or otherwise, about the performance or reliability of these products.