Application Gateway Incress Controller with Kubernetes - IPs not Updating after HTTPs Listener and Rule

Ryan O'Donnell 1 Reputation point
2022-02-16T21:35:36.02+00:00

Hi,

Hitting a really odd problem. I am doing exactly as described here: https://learn.microsoft.com/en-us/azure/application-gateway/tutorial-ingress-controller-add-on-new

But with my own app. My app is pretty simple, and the ingress is exactly the same as above (but I use the name of my service not the name of their service).

This deploys and works great - but the IPs for the backend pool only update before I add a HTTPS listener and rule. Once I add the listener and rule, and cycle the pods on k8s, it takes a long time for the health checks to fail, and even after they fail the backend pool IPs do not update on the application gateway.

I've reproduced this a few times. Delete the kubernetes cluster on azure, deploy using the below command (essentially from the above tutorial, only thing added is the azure registry connection) wait for everything to go through and the gateway to get an IP, deploy my config and updating the image on kubernetes updates backend pool IPs, as soon as I add a listener and rule the application gateway breaks. Same thing each time.

Crazy thing is it also seems to completely break the AG. As in, even after I remove the rule and listener it does not start working. I have tried deleting the ingress and reapplying and that doesn't work either.

I can see events in the activity for the AG from the "ingressapplicationgateway-<name>" just stop coming through.

az aks create -n <aks-name> -g <RG> --network-plugin azure --enable-managed-identity -a ingress-appgw --appgw-name <agw-name> --appgw-subnet-cidr "10.2.0.0/16" --generate-ssh-keys --attach-acr <acr-name>

Describe ingress doesn't give me any events.

Kubernetes config with my stuff stripped:

apiVersion: apps/v1
kind: Deployment
metadata:
name: <name>
labels:
app: <name>
spec:
replicas: 3
selector:
matchLabels:
app: <name>
strategy:
type: RollingUpdate
template:
metadata:
labels:
app: <name>
spec:
containers:

  • name: <name>
    imagePullPolicy: "Always"
    image: <private registry image>
    ports:
  • containerPort: 80
    command: [<some-command>]
    args: ['some-args']

apiVersion: v1
kind: Service
metadata:
name: <name>
labels:
app: <name>
spec:
type: LoadBalancer
ports:

  • port: 80
    protocol: TCP
    name: <name>
    selector:
    app: <name>

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: <some-ingress-name>
annotations:
kubernetes.io/ingress.class: azure/application-gateway
spec:
rules:

  • http:
    paths:
  • path: /
    backend:
    service:
    name: <service-name>
    port:
    number: 80
    pathType: Exact
Azure Application Gateway
Azure Application Gateway
An Azure service that provides a platform-managed, scalable, and highly available application delivery controller as a service.
1,045 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
2,073 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.