Could not provisioned AKS on Azure Stack HCI cluster using version 23H2?

Question

I have a failed AKS on 23H2 Azure Stack HCI cluster created using command: az aksarc create -n $aksclustername -g $resource-group --custom-location $customLocationID --vnet-ids $lnetid --aad-admin-group-object--ids $aadgroupID --generate-ssh-keys --load-balancer-count 0 --control-plane-ip $controlplaneIP --verbose Error: BadRequest Detail: admission webhook "vhybridakscluster.kb.io" denied the request: Kubernetes version 1.27.3 is not ready for use on linux The aks arc resource is showing on Azure Portal with provisioning State = failed

Answer 1

Update to this case: After working with Microsoft support, it turns out that I have problem with the custom location being offline. We restarted the Arc Resource Bridge appliance VM and it came back online. Then we delete the failed aks arc cluster with az aksarc delete --name $aksclustername --resource_group --yes. It successfully deleted the failed cluster, previously even deleting it was failing. Now we try to create new one again but failed. We are still investigating it though.

Answer 2

I don't have the fix yet but I was working with our partner and were able to get a answer from Microsoft. This error typically means the AKS image failed to download, the command to run is "az aksarc get-versions --custom-location <cl> --resource-group <rg>". I ran this and was able to see the following error message, among other.

           "errorMessage": "Image is waiting to be provisioned",
            "osSku": "CBLMariner",
            "osType": "Linux",
            "ready": false  

If I get an answer on a fix I will try to post it here.