How to run kubectl for Private AKS cluster from user laptop

Question

Hi Team,

I have created private cluster by following the below link. Am able to access the cluster from my laptop using the option of "AKS Run Command".

https://learn.microsoft.com/en-us/azure/aks/private-clusters

Now, i need to create nginx ingress controller using helm chart. When i ran any helm command, it gives the permission error

Please help me to solve this issue or any other way to create the ingress controller for private AKS.

az aks command invoke -g private-aks -n aks-demo -c "helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx" command started at 2021-05-28 00:09:10+00:00, finished at 2021-05-28 00:09:10+00:00, with exitcode=1 Error: mkdir /.config: permission denied

Thanks
Raam

Answer 1

Hello @CHANEMOUGAM, RAAMACHANTHIRAN ,
Thanks for your query.
I was able to repro the issue and below are the resolution steps which I followed:

1) Create a Azure Linux VM in the same network as that of Private AKS cluster
2) Install azure CLI package in the Linux Azure VM
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
3) Install helm :
Follow the steps mentioned here:
https://phoenixnap.com/kb/install-helm#:~:text=To%20install%20Helm%20on%20Ubuntu%3A%201%20Download%20the,Helm%20by%20checking%20the%20version%20of%20the%20software%3A

wget https://get.helm.sh/helm-v3.4.1-linux-amd64.tar.gz
sudo mv linux-amd64/helm /usr/local/bin
Remove downloaded file
rm helm-v3.4.1-linux-amd64.tar.gz
clean up space by removing extracted file
rm -rf linux-amd64

shpatpi@linuxmc:~$ helm version
version.BuildInfo{Version:"v3.4.1", GitCommit:"c4e74854886b2efe3321e185578e6db9be0a6e29", GitTreeState:"clean", GoVersion:"go1.14.11"}

shpatpi@linuxmc:~$ az aks get-credentials -g privateakscluster -n privateakscluster
Merged "privateakscluster" as current context in /home/shpatpi/.kube/config

shpatpi@linuxmc:~$ helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
"ingress-nginx" has been added to your repositories

shpatpi@linuxmc:~$ az aks command invoke -g privateakscluster -n privateakscluster -c "kubectl get ns"
command started at 2021-05-28 04:46:31+00:00, finished at 2021-05-28 04:46:31+00:00 with exitcode=0
NAME STATUS AGE
aks-command Active 3h13m
default Active 3h21m
ingress-basic Active 178m
kube-node-lease Active 3h21m
kube-public Active 3h21m
kube-system Active 3h21m

Install ingress controller

shpatpi@linuxmc:~$ helm install nginx-ingress ingress-nginx/ingress-nginx --namespace ingress-basic --set controller.replicaCount=2 --set controller.nodeSelector."beta.kubernetes.io/os"=linux --set defaultBackend.nodeSelector."beta.kubernetes.io/os"=linux --set controller.admissionWebhooks.patch.nodeSelector."beta.kubernetes.io/os"=linux NAME: nginx-ingress

Validation of installation:

shpatpi@linuxmc:~$ az aks command invoke -g privateakscluster -n privateakscluster -c "kubectl get pods -n ingress-basic"
command started at 2021-05-28 04:54:44+00:00, finished at 2021-05-28 04:54:44+00:00 with exitcode=0
NAME READY STATUS RESTARTS AGE
nginx-ingress-ingress-nginx-controller-85484674f4-69q8b 1/1 Running 0 7m50s
nginx-ingress-ingress-nginx-controller-85484674f4-fbtm8 1/1 Running 0 7m50s

Above steps should help you out in resolving the issue , Kindly make sure to "Upvote and Accept the Answer" so that it can be helpful to the community out there.

Answer 2

Hello @CHANEMOUGAM, RAAMACHANTHIRAN ,
You won't be able to access the private cluster from your laptop and the error which you are getting is expected.

When you create a Private AKS Cluster , The API server endpoint has no public IP address.
To manage the API server, you'll need to use a VM that has access to the AKS cluster's Azure Virtual Network (VNet).

There are several options for establishing network connectivity to the private cluster.

• Create a VM in the same Azure Virtual Network (VNet) as the AKS cluster.
• Use a VM in a separate network and set up Virtual network peering. See the section below for more information on this option.
• Use an Express Route or VPN connection.
• Use the AKS Run Command feature. (It will not work for helm based deployments)
• Creating a VM in the same VNET as the AKS cluster is the easiest option. Express Route and VPNs add costs and require additional networking complexity. Virtual network peering requires you to plan your network CIDR ranges to ensure there are no overlapping ranges

So, you have to go with the options which I posted earlier.

Kindly let me know if you have additional questions. If those answers helps you out don't forget to "Upvote and Accept the answer"

Answer 3

@shiva patpi

I got the jump box VM configured on the same vnet. Now when i tried to run any kubectl commands got the below error.

[XXXX@mulesoft-jump-server bin]$kubectl get services --all-namespaces
an error on the server ("") has prevented the request from succeeding

Any help in resolving the issue would be much appreciated

Thanks
Raam