AKS Service deployment done and can´t reach external IPs for specific service

Question

I deployed two services with azure AKS and cant reach external ip of a specific service, both services are relying on linux containers. I find strange that service pods can run the docker web service via port fowarding, and also have access to internet when I go inside do a curl request. Some nodes access inside the pods arent reacheable, but one pod can reach other pod in the same service.

Answer 1

I'm facing the same issue. I'm working on the following MS Learn exercise: Deploy a containerized application on Azure Kubernetes Service (https://learn.microsoft.com/en-us/training/modules/aks-deploy-container-app/7-exercise-expose-app). After completing the last exercise, I'm unable to reach my application's website. Not sure where to look for troubleshooting, all the health checks in Azure are passing.

Shahab

Answer 2

Hi @AKS-MON-01 ,

Can you validate in your environment if your service type is LoadBalancer?

To expose a service in AKS to external world you have two options, expose each pod/deployment via one respective service or create a Ingress Controller (This is work like a Reverse Proxy) and expose each itnernal service via Ingress Controller.

You can find more information at the links:

https://learn.microsoft.com/en-us/azure/aks/load-balancer-standard
https://learn.microsoft.com/en-us/azure/aks/ingress-basic

Answer 3

thanks for your reply @Lucas Camargo Reis .

Both are load balancers, the working one and the not working one.
I find strange that one is working without any specific settings and the other doesn´t.
I haven´t made any specific settings for outbound ips and ingress for one to work.

Answer 4

Could you send the yaml files used to deploy services and the output for the comands:

1. kubectl get svc <servicename> -o yaml
2. kubectl get svc <servicename> -o wide
3. kubectl describe svc <servicename>

Answer 5

service deployment YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  labels:
    environment: production
    app: my-app
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: my-app
    spec:
      nodeSelector:
          "beta.kubernetes.io/os": linux
      containers:
      \- env:
         \- name: PUBLIC_FOLDER
           value: ../site
         \- name: HTTP_PORT
           value: "80"
        image: <registry_url>/<registry_user>/<name_image>
        imagePullPolicy: Always
        name: my-app
        ports:
        \- containerPort: 80
        readinessProbe:
          httpGet:
            port: 80
            path: /api/
        livenessProbe:
          httpGet:
            port: 80
            path: /api/
      imagePullSecrets:
        \- name: regcred
  selector:
    matchLabels:
      app: my-app
---
apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  type: "LoadBalancer"
  externalTrafficPolicy: Local
  ports:
  \- name: "http"
    protocol: TCP
    port: 80
    targetPort: 80
  selector:
    name: my-app

Service YAML

apiVersion: v1 kind: Service metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"my-app","namespace":"default"},"spec":{"externalTrafficPolicy":"Local","ports":[{"name":"http","port":80,"protocol":"TCP","targetPort":80}],"selector":{"name":"my-app"},"type":"LoadBalancer"}} creationTimestamp: "2021-02-19T20:00:00Z" finalizers: - service.kubernetes.io/load-balancer-cleanup managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:annotations: .: {} f:kubectl.kubernetes.io/last-applied-configuration: {} f:spec: f:externalTrafficPolicy: {} f:ports: .: {} k:{"port":80,"protocol":"TCP"}: .: {} f:name: {} f:port: {} f:protocol: {} f:targetPort: {} f:selector: .: {} f:name: {} f:sessionAffinity: {} f:type: {} manager: kubectl-client-side-apply operation: Update time: "2021-02-19T20:00:00Z" - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: {} v:"service.kubernetes.io/load-balancer-cleanup": {} f:status: f:loadBalancer: f:ingress: {} manager: kube-controller-manager operation: Update time: "2021-02-19T20:00:15Z" name: my-app namespace: default resourceVersion: "3781154" selfLink: /api/v1/namespaces/default/services/my-app uid: f5261e77-61dc-4fa3-822d-ed178aef0851 spec: clusterIP: 10.0.246.3 externalTrafficPolicy: Local healthCheckNodePort: 31524 ports: - name: http nodePort: 31635 port: 80 protocol: TCP targetPort: 80 selector: name: my-app sessionAffinity: None type: LoadBalancer status: loadBalancer: ingress: - ip: 52.149.58.105

Service wide

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR my-app LoadBalancer 10.0.246.3 52.149.58.105 80:31635/TCP 3m30s name=my-app

Service Describe

Name: my-app Namespace: default Labels: <none> Annotations: <none> Selector: name=my-app Type: LoadBalancer IP: 10.0.246.3 LoadBalancer Ingress: 52.149.58.105 Port: http 80/TCP TargetPort: 80/TCP NodePort: http 31635/TCP Endpoints: <none> Session Affinity: None External Traffic Policy: Local HealthCheck NodePort: 31524 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal EnsuringLoadBalancer 3m38s service-controller Ensuring load balancer Normal EnsuredLoadBalancer 3m23s service-controller Ensured load balancer

Working Service Deployment YAML

apiVersion: apps/v1 kind: Deployment metadata: name: azure-vote-back spec: replicas: 1 selector: matchLabels: app: azure-vote-back template: metadata: labels: app: azure-vote-back spec: nodeSelector: "beta.kubernetes.io/os": linux containers: - name: azure-vote-back image: mcr.microsoft.com/oss/bitnami/redis:6.0.8 env: - name: ALLOW_EMPTY_PASSWORD value: "yes" resources: requests: cpu: 100m memory: 128Mi limits: cpu: 250m memory: 256Mi ports: - containerPort: 6379

name: redis

apiVersion: v1 kind: Service metadata: name: azure-vote-back spec: ports: - port: 6379 selector:

app: azure-vote-back

apiVersion: apps/v1 kind: Deployment metadata: name: azure-vote-front spec: replicas: 1 selector: matchLabels: app: azure-vote-front template: metadata: labels: app: azure-vote-front spec: nodeSelector: "beta.kubernetes.io/os": linux containers: - name: azure-vote-front image: mcr.microsoft.com/azuredocs/azure-vote-front:v1 resources: requests: cpu: 100m memory: 128Mi limits: cpu: 250m memory: 256Mi ports: - containerPort: 80 env: - name: REDIS

value: "azure-vote-back"

apiVersion: v1 kind: Service metadata: name: azure-vote-front spec: type: LoadBalancer ports: - port: 80 selector: app: azure-vote-front

Working Services yaml

apiVersion: v1 kind: Service metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"azure-vote-front","namespace":"default"},"spec":{"externalIPs":["20.190.19.155"],"ports":[{"port":80}],"selector":{"app":"azure-vote-front"},"type":"LoadBalancer"}} creationTimestamp: "2021-02-09T17:36:22Z" finalizers: - service.kubernetes.io/load-balancer-cleanup managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: {} v:"service.kubernetes.io/load-balancer-cleanup": {} f:status: f:loadBalancer: f:ingress: {} manager: kube-controller-manager operation: Update time: "2021-02-09T17:36:27Z" - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:annotations: .: {} f:kubectl.kubernetes.io/last-applied-configuration: {} f:spec: f:externalIPs: {} f:externalTrafficPolicy: {} f:ports: .: {} k:{"port":80,"protocol":"TCP"}: .: {} f:port: {} f:protocol: {} f:targetPort: {} f:selector: .: {} f:app: {} f:sessionAffinity: {} f:type: {} manager: kubectl-client-side-apply operation: Update time: "2021-02-16T18:34:59Z" name: azure-vote-front namespace: default resourceVersion: "3151634" selfLink: /api/v1/namespaces/default/services/azure-vote-front uid: eeebb173-49a1-4d41-94a5-a4058e30f4f4 spec: clusterIP: 10.0.50.241 externalIPs: - 20.190.19.155 externalTrafficPolicy: Cluster ports: - nodePort: 31317 port: 80 protocol: TCP targetPort: 80 selector: app: azure-vote-front sessionAffinity: None type: LoadBalancer status: loadBalancer: ingress: - ip: 20.190.19.152

apiVersion: v1 kind: Service metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"azure-vote-back","namespace":"default"},"spec":{"ports":[{"port":6379}],"selector":{"app":"azure-vote-back"}}} creationTimestamp: "2021-02-09T17:36:21Z" managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:annotations: .: {} f:kubectl.kubernetes.io/last-applied-configuration: {} f:spec: f:ports: .: {} k:{"port":6379,"protocol":"TCP"}: .: {} f:port: {} f:protocol: {} f:targetPort: {} f:selector: .: {} f:app: {} f:sessionAffinity: {} f:type: {} manager: kubectl-client-side-apply operation: Update time: "2021-02-09T17:36:21Z" name: azure-vote-back namespace: default resourceVersion: "1697371" selfLink: /api/v1/namespaces/default/services/azure-vote-back uid: ab864d8d-a0b8-4b85-82ae-fe3007f83e2c spec: clusterIP: 10.0.85.65 ports: - port: 6379 protocol: TCP targetPort: 6379 selector: app: azure-vote-back sessionAffinity: None type: ClusterIP status: loadBalancer: {}

Working Service Wide

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR azure-vote-front LoadBalancer 10.0.50.241 20.190.19.152,20.190.19.155 80:31317/TCP 10d app=azure-vote-front

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR azure-vote-back ClusterIP 10.0.85.65 <none> 6379/TCP 10d app=azure-vote-back

Working Service Describe

Name: azure-vote-front Namespace: default Labels: <none> Annotations: <none> Selector: app=azure-vote-front Type: LoadBalancer IP: 10.0.50.241 External IPs: 20.190.19.155 LoadBalancer Ingress: 20.190.19.152 Port: <unset> 80/TCP TargetPort: 80/TCP NodePort: <unset> 31317/TCP Endpoints: 10.244.1.9:80 Session Affinity: None External Traffic Policy: Cluster Events: <none>

Name: azure-vote-back Namespace: default Labels: <none> Annotations: <none> Selector: app=azure-vote-back Type: ClusterIP IP: 10.0.85.65 Port: <unset> 6379/TCP TargetPort: 6379/TCP Endpoints: 10.244.1.8:6379 Session Affinity: None Events: <none>