How to run kubectl for Private AKS cluster from user laptop

CHANEMOUGAM, RAAMACHANTHIRAN 26 Reputation points
2021-05-28T00:38:00.773+00:00

Hi Team,

I have created private cluster by following the below link. Am able to access the cluster from my laptop using the option of "AKS Run Command".

https://learn.microsoft.com/en-us/azure/aks/private-clusters

Now, i need to create nginx ingress controller using helm chart. When i ran any helm command, it gives the permission error

Please help me to solve this issue or any other way to create the ingress controller for private AKS.

az aks command invoke -g private-aks -n aks-demo -c "helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx" command started at 2021-05-28 00:09:10+00:00, finished at 2021-05-28 00:09:10+00:00, with exitcode=1 Error: mkdir /.config: permission denied

Thanks
Raam

Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,878 questions

3 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. shiva patpi 13,141 Reputation points Microsoft Employee
    2021-05-28T05:00:22.897+00:00

    Hello @CHANEMOUGAM, RAAMACHANTHIRAN ,
    Thanks for your query.
    I was able to repro the issue and below are the resolution steps which I followed:

    1) Create a Azure Linux VM in the same network as that of Private AKS cluster
    2) Install azure CLI package in the Linux Azure VM
    curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
    3) Install helm :
    Follow the steps mentioned here:
    https://phoenixnap.com/kb/install-helm#:~:text=To%20install%20Helm%20on%20Ubuntu%3A%201%20Download%20the,Helm%20by%20checking%20the%20version%20of%20the%20software%3A

    wget https://get.helm.sh/helm-v3.4.1-linux-amd64.tar.gz
    sudo mv linux-amd64/helm /usr/local/bin
    Remove downloaded file
    rm helm-v3.4.1-linux-amd64.tar.gz
    clean up space by removing extracted file
    rm -rf linux-amd64

    shpatpi@linuxmc:~$ helm version
    version.BuildInfo{Version:"v3.4.1", GitCommit:"c4e74854886b2efe3321e185578e6db9be0a6e29", GitTreeState:"clean", GoVersion:"go1.14.11"}

    shpatpi@linuxmc:~$ az aks get-credentials -g privateakscluster -n privateakscluster
    Merged "privateakscluster" as current context in /home/shpatpi/.kube/config

    shpatpi@linuxmc:~$ helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
    "ingress-nginx" has been added to your repositories

    shpatpi@linuxmc:~$ az aks command invoke -g privateakscluster -n privateakscluster -c "kubectl get ns"
    command started at 2021-05-28 04:46:31+00:00, finished at 2021-05-28 04:46:31+00:00 with exitcode=0
    NAME STATUS AGE
    aks-command Active 3h13m
    default Active 3h21m
    ingress-basic Active 178m
    kube-node-lease Active 3h21m
    kube-public Active 3h21m
    kube-system Active 3h21m

    Install ingress controller

    shpatpi@linuxmc:~$ helm install nginx-ingress ingress-nginx/ingress-nginx --namespace ingress-basic --set controller.replicaCount=2 --set controller.nodeSelector."beta.kubernetes.io/os"=linux --set defaultBackend.nodeSelector."beta.kubernetes.io/os"=linux --set controller.admissionWebhooks.patch.nodeSelector."beta.kubernetes.io/os"=linux NAME: nginx-ingress

    Validation of installation:

    shpatpi@linuxmc:~$ az aks command invoke -g privateakscluster -n privateakscluster -c "kubectl get pods -n ingress-basic"
    command started at 2021-05-28 04:54:44+00:00, finished at 2021-05-28 04:54:44+00:00 with exitcode=0
    NAME READY STATUS RESTARTS AGE
    nginx-ingress-ingress-nginx-controller-85484674f4-69q8b 1/1 Running 0 7m50s
    nginx-ingress-ingress-nginx-controller-85484674f4-fbtm8 1/1 Running 0 7m50s

    Above steps should help you out in resolving the issue , Kindly make sure to "Upvote and Accept the Answer" so that it can be helpful to the community out there.

    1 person found this answer helpful.
  2. shiva patpi 13,141 Reputation points Microsoft Employee
    2021-06-09T13:30:26.01+00:00

    Hello @CHANEMOUGAM, RAAMACHANTHIRAN ,
    You won't be able to access the private cluster from your laptop and the error which you are getting is expected.

    When you create a Private AKS Cluster , The API server endpoint has no public IP address.
    To manage the API server, you'll need to use a VM that has access to the AKS cluster's Azure Virtual Network (VNet).

    There are several options for establishing network connectivity to the private cluster.

    • Create a VM in the same Azure Virtual Network (VNet) as the AKS cluster.
    • Use a VM in a separate network and set up Virtual network peering. See the section below for more information on this option.
    • Use an Express Route or VPN connection.
    • Use the AKS Run Command feature. (It will not work for helm based deployments)
    • Creating a VM in the same VNET as the AKS cluster is the easiest option. Express Route and VPNs add costs and require additional networking complexity. Virtual network peering requires you to plan your network CIDR ranges to ensure there are no overlapping ranges

    So, you have to go with the options which I posted earlier.

    Kindly let me know if you have additional questions. If those answers helps you out don't forget to "Upvote and Accept the answer"

    1 person found this answer helpful.
  3. CHANEMOUGAM, RAAMACHANTHIRAN 26 Reputation points
    2021-06-24T04:59:51.607+00:00

    @shiva patpi

    I got the jump box VM configured on the same vnet. Now when i tried to run any kubectl commands got the below error.

    [XXXX@mulesoft-jump-server bin]$kubectl get services --all-namespaces
    an error on the server ("") has prevented the request from succeeding

    Any help in resolving the issue would be much appreciated

    Thanks
    Raam