This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 11%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Hi All,
My cluster is in failed state. upon observing the cluster I found out that a node pool is in failed state so I have drained the nodes one by one and deleted the node pool. its been 12 hours but the cluster still shows that it is in failed state
can anyone please help me with this
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 98%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
Let me know the results and I can further look into it. Thanks.
Hi, I am not seeing the answers in your comment. Can you please post again. Also, I updated my comment with few more points, please check. Thanks.
yes , I have seen it now. I will update you in a bit. Thanks
initially, I wanted to do the cluster upgrade so I have been visting the cluster from past few days. from few days, its been in failed state. as per the screenshot, I thought it will get resolve on its own since I have not performed any action. so I visited it yesterday. it was still in failed state. Then, I have seen the one of the node pool is in failed state because of the insufficient IP addresses . Since this nodepool is not needed, I decided to delete the nodepool. I have followed below commands and deleted the nodes 1 after the other then I have deleted the nodepool
kubectl drain < node name> --ignore-daemonsets --delete-local-data
kubectl delete node < nodename>
no. I did not start the operation since the option is disabled for me
can you please be more specific here. is it the activity log you are asking
it is redirecting me to this page - > https://learn.microsoft.com/en-us/troubleshoot/azure/azure-kubernetes/cluster-provisioning-status-changed-ready-failed
I did not initiate the cluster. but as doc suggested I have seen the quota limits ( I dnt think the limit is excceded) and executed the command shown . its giving me following results.
will network watcher have effect?
Let me know the results and I can further look into it. Thanks.
no we are not using it. we are using istio ingress controller to route the traffic. istio version in 1.5.2 cluster version 1.24.0
Hi @KarishmaTiwari-MSFT ,
FYI these are the suggestions I am getting
but this node and node-pool were already deleted
I am not sure why this error getting now
Hi ,
I have tried this command and it resolved the issue.
upgrading the cluster to the current version ( my cluster in on 1.24.0 so trying to upgrade to the same version)
az aks upgrade --name <cluster name> --resource-group rg --subscription "subscrition" --kubernetes-version 1.24.0 --yes
I have tried this command and it resolved the issue.
upgrading the cluster to the current version ( my cluster in on 1.24.0 so trying to upgrade to the same version)
az aks upgrade --name <cluster name> --resource-group rg --subscription "subscrition" --kubernetes-version 1.24.0 --yes
Thanks for sharing the update and resolution with others in the community.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 37%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPA%3C/text%3E%3C/svg%3E)
I'm in the same situation, and I can't update aks because it is in a "failed" state because the version that my nodepool has is outdated. I can't update it because my aks is "failed". So I find myself in an impossible circle to solve.
az aks upgrade --resource-group <resource-group> --name <aks-name> --kubernetes-version 1.25.11 --yes
(OperationNotAllowed) Upgrades are disallowed while cluster is in a failed state. For resolution steps visit https://aka.ms/aks-cluster-failed to troubleshoot why the cluster state may have failed and steps to fix cluster state.Code: OperationNotAllowedMessage: Upgrades are disallowed while cluster is in a failed state. For resolution steps visit https://aka.ms/aks-cluster-failed to troubleshoot why the cluster state may have failed and steps to fix cluster state.