Cannot delete Application Gateway for Containers

Question

Hello,

We're using Application Gateway for Containers created and managed by the ALB controller. I need to delete the AGC used for testing, but I'm unable to do so completely. Specifically, I cannot delete the ApplicationLoadBalancer and the GatewayClass resources in AKS. When I execute the kubectl delete command for these resources, it indicates that the resource has been deleted, but the terminal gets stuck, and the prompt doesn't appear. After about 30 minutes or more, the following error messages are displayed:

kubectl delete applicationloadbalancer alb-test -n agc-test
applicationloadbalancer.alb.networking.azure.io "alb-test" deleted
W0430 17:11:52.338083    8952 reflector.go:470] k8s.io/client-go/tools/watch/informerwatcher.go:146: watch of *unstructured.Unstructured ended with: an error on the server ("unable to decode an event from the watch stream: read tcp 192.168.50.210:60177->52.188.44.19:443: wsarecv: An established connection was aborted by the software in your host machine.") has prevented the request from succeeding
W0430 17:11:53.215157    8952 reflector.go:547] k8s.io/client-go/tools/watch/informerwatcher.go:146: failed to list *unstructured.Unstructured: Get "https://akshosteus-f85425b7.hcp.eastus.azmk8s.io:443/apis/alb.networking.azure.io/v1/namespaces/agc-test/applicationloadbalancer?fieldSelector=metadata.name%3Dalb-test&resourceVersion=845538588": dial tcp: lookup akshosteus-f85425b7.hcp.eastus.azmk8s.io: no such host
E0430 17:11:53.215671    8952 reflector.go:150] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to watch *unstructured.Unstructured: failed to list *unstructured.Unstructured: Get "https://akshosteus-f85425b7.hcp.eastus.azmk8s.io:443/apis/alb.networking.azure.io/v1/namespaces/agc-test/applicationloadbalancer?fieldSelector=metadata.name%3Dalb-test&resourceVersion=845538588": dial tcp: lookup akshosteus-f85425b7.hcp.eastus.azmk8s.io: no such host
W0430 17:11:56.030272    8952 reflector.go:547] k8s.io/client-go/tools/watch/informerwatcher.go:146: failed to list *unstructured.Unstructured: Get "https://akshosteus-f85425b7.hcp.eastus.azmk8s.io:443/apis/alb.networking.azure.io/v1/namespaces/agc-test/applicationloadbalancer?fieldSelector=metadata.name%3Dalb-test&resourceVersion=845538588": dial tcp: lookup akshosteus-f85425b7.hcp.eastus.azmk8s.io: no such host
E0430 17:11:56.030805    8952 reflector.go:150] k8s.io/client-go/tools/watch/informerwatcher.go:146: Failed to watch *unstructured.Unstructured: failed to list *unstructured.Unstructured: Get "https://akshosteus-f85425b7.hcp.eastus.azmk8s.io:443/apis/alb.networking.azure.io/v1/namespaces/agc-test/applicationloadbalancer?fieldSelector=metadata.name%3Dalb-test&resourceVersion=845538588": dial tcp: lookup akshosteus-f85425b7.hcp.eastus.azmk8s.io: no such host
Error: failed to get token: initialing the device code authentication: autorest/adal/devicetoken: Error occurred while sending request for Device Authorization Code: Post "https://login.microsoftonline.com/62b23027-b47b-4096-8b99-4f3185889be5/oauth2/devicecode": dial tcp: lookup login.microsoftonline.com: no such host

I can delete the Application Gateway for Containers using the Azure portal, but it doesn't remove the underlying ApplicationLoadBalancer resource from AKS; it's still there.

Instead of the local terminal, I tried the Cloud shell as well, but the result is still the same. Also tried to uninstall the ALB controller from the AKS, but no effect.

Is there any recommendation on how to delete AGC from Azure/AKS, or what I'm doing wrong?

Thank you.

Answer 1

Hi Kamil Grebenicek,

I understand that you are trying to delete the Application Gateway for Containers and the associated Application LoadBalancer resources but leading to connection errors. This may happen because of the Finalizers.

Kubernetes resources sometimes get stuck in a Terminating state due to lingering finalizers that prevent deletion. Finalizers are special metadata fields that tell Kubernetes to wait before deleting an object until certain cleanup operations are completed. They act as pre-delete hooks, ensuring that dependent resources are properly handled before Kubernetes removes an object.

You can check if there is a finalizer for the Application Load balancer with the command below.

kubectl get applicationloadbalancer alb-test -n agc-test -o json > alb.json

If there is a Finalizer, you can remove it by editing the file. Then apply it with, this removes the application Load balancer.

kubectl replace --raw "/apis/alb.networking.azure.io/v1/namespaces/agc-test/applicationloadbalancers/alb-test/finalize" -f alb.json

Similarly, please check for the GatewayClass, if there is a finalizer.

Please refer to the documents about Finalizers and using-finalizers-to-control-deletion

If you have any further queries, let me know. If the information is helpful, please click on Upvote.

Thank you

Answer 2

Hello Kamil Grebenicek,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you cannot delete Application Gateway for Containers in your explanation.

1. Before removing custom resources like ApplicationLoadBalancer, you must uninstall the ALB controller cleanly. Otherwise, finalizers may block the deletion.
2. Cleanly Uninstall the ALB Controller by using Helm or the method you originally used to install the ALB controller. For Helm use bash command: helm uninstall alb-controller -n kube-system Wait 2–3 minutes. Then check if the controller pods are gone using bash command: kubectl get pods -n kube-system | grep alb So, if ALB controller is still running, the finalizers may not be cleaned properly. A running controller is required to remove its own finalizers.
3. If the controller has been uninstalled and resources are stuck, do this carefully using bash: kubectl get applicationloadbalancer alb-test -n agc-test -o json > alb.json Edit the alb.json file: Remove the "finalizers" block inside metadata Then: kubectl replace --raw "/apis/alb.networking.azure.io/v1/namespaces/agc-test/applicationloadbalancers/alb-test/finalize" -f alb.json Repeat similar steps for GatewayClass.
4. Also, some AGC components create Azure infrastructure (e.g., App Gateway, Public IPs, NICs). Use these commands:

      az network application-gateway list -o table
      az network public-ip list -o table
      az network nic list --query "[?contains(name, 'alb-test')]" -o table
   

   Manually delete orphaned Azure resources:

      az network application-gateway delete --name <gateway-name> --resource-group <rg>
   

5. Should there be any errors such as: lookup akshosteus-f85425b7.hcp.eastus.azmk8s.io: no such hostsuggest host DNS issues. Resolve this by:
   • Restarting your local system's DNS resolver
   • Switching to a different machine
   • Using a working VPN or private DNS if AKS is in a private cluster
   • Trying a different node via: kubectl config use-context <aks-context>
   • Or try deletion from Azure Cloud Shell, if local networking is unstable.
6. Check that CRDs are removed by using bash:
   • kubectl get applicationloadbalancer -A
   • kubectl get gatewayclass

I hope this is helpful! Do not hesitate to let me know if you have any other questions or clarifications.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.