Terraform Helm Provider Error - "Kubernetes cluster unreachable" in AKS

Question

I am trying to use Terraform to deploy Helm charts on an Azure Kubernetes Service (AKS) cluster. However, when I apply the Terraform configuration, I get the following error:

helm_release.blackbox-exporter: Creating...
╷
│ Error: Kubernetes cluster unreachable: the server has asked for the client to provide credentials
│
│   with helm_release.blackbox-exporter,
│   on test.tf line 1, in resource "helm_release" "blackbox-exporter":
│    1: resource "helm_release" "blackbox-exporter" {
│
╵

Following is the Terraform code that I am using for helm provider and AKS data source:

provider "helm" {
  kubernetes {
    host                   = data.azurerm_kubernetes_cluster.this.kube_config[0].host
    client_certificate     = base64decode(data.azurerm_kubernetes_cluster.this.kube_config[0].client_certificate)
    client_key            = base64decode(data.azurerm_kubernetes_cluster.this.kube_config[0].client_key)
    cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.this.kube_config[0].cluster_ca_certificate)
  }
}

data "azurerm_kubernetes_cluster" "this" {
  name                = local.aks_cluster_name
  resource_group_name = local.aks_resource_group
}

I am using AKS with k8s version 1.32 and MS Entra ID authentication with Azure RBAC. Can you anyone please help me understand what is the issue here?

Answer 1

Hello Mahesh,

Thank you for posting your question in the Microsoft Q&A forum.

The error message Kubernetes cluster unreachable: the server has asked for the client to provide credentials indicates that Terraform is unable to authenticate with your Azure Kubernetes Service (AKS) cluster. This issue commonly arises when the Kubernetes cluster is configured with Microsoft Entra ID (formerly Azure Active Directory) authentication and Azure RBAC, as these configurations require additional steps to authenticate with the cluster.

When using Microsoft Entra ID authentication with AKS, the kubeconfig file generated by Azure does not include static credentials (like client certificates or keys). Instead, it relies on Azure CLI or other authentication mechanisms to dynamically fetch credentials. The Terraform Helm provider, as configured in your code, attempts to use static credentials from the kubeconfig, which are not available in this setup.

Since your AKS cluster uses Microsoft Entra ID authentication, you need to authenticate Terraform using Azure CLI. The Helm provider can leverage the Azure CLI to fetch the necessary credentials dynamically. Update your Terraform configuration to use the exec block in the Helm provider, which allows you to run a command (like az aks get-credentials) to fetch the kubeconfig dynamically

The issue arises because the Helm provider is attempting to use static credentials, which are not available when using Microsoft Entra ID authentication with AKS. By leveraging the Azure CLI for dynamic authentication and configuring the Helm provider to use the exec block, you can resolve the issue. Additionally, ensure that the Azure account used for authentication has the necessary permissions to access the AKS cluster.

 

Some documentation you may refer:

• https://learn.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-get-credentials
• https://learn.microsoft.com/en-us/azure/aks/enable-authentication-microsoft-entra-id
• https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs#exec-plugins
• https://learn.microsoft.com/en-us/azure/aks/learn/quick-kubernetes-deploy-terraform?pivots=development-environment-azure-cli

 

If the above answer helped, please do not forget to "Accept Answer" as this may help other community members to refer the info if facing a similar issue. Your contribution to the Microsoft Q&A community is highly appreciated.