HPA is not working properly in AKS

Question

Hi team,
I have deployed an HPA and defined the metrics for the CPU and memory, in case of high load the pod as well as the cluster is scaling but when the load reduces they are not scaling down. Please find the below details

error on the HPA

 Warning  FailedGetResourceMetric       8m2s (x22 over 111m)  horizontal-pod-autoscaler  failed to get cpu utilization: did not receive metrics for any ready pods     
  Warning  FailedComputeMetricsReplicas  8m2s (x8 over 54m)    horizontal-pod-autoscaler  invalid metrics (1 invalid out of 2), first error is: failed to get cpu utilization: did not receive metrics for any ready pods     

HPA deployment file

apiVersion: autoscaling/v2beta2     
kind: HorizontalPodAutoscaler     
metadata:     
  name: dev-hpa     
  namespace: dev-iapi     
spec:     
  maxReplicas: 5 # define max replica count     
  minReplicas: 2  # define min replica count     
  scaleTargetRef:     
    apiVersion: apps/v1     
    kind: Deployment     
    name: dev-deployment     
  metrics:     
    - type: Resource     
      resource:     
        name: cpu     
        target:     
          type: Utilization     
          averageUtilization: 70     
    - type: Resource     
      resource:     
        name: memory     
        target:     
          type: Utilization     
          averageUtilization: 65     

Answer 1

I am also facing the same issue in AKS, Could you please some one suggest us on this issue.

Answer 2

HI ,

i found the probleme. The label i used for the lbtype

apiVersion: apps/v1
kind: Deployment
metadata:
name: ur-front
namespace: recette-ur
labels:
app: front
spec:
replicas: 1
selector:
matchLabels:
lbtype: external
app: front
template:
metadata:
labels:
lbtype: external
app: front

the label in bold was missing

Answer 3

Hi , im using aks version 1.24.3

Answer 4

i am also seeing similar behavior of error messages. It starting to become a big problem for your organisation , I had to static number of pods because of this (autoscaling not possible anymore)

Answer 5

I'm having this same problem, I'll follow this post to see if anyone found the solution.