Share via

Health Check Frequency Exceedingly High

Matt Robold (WCSG) 96 Reputation points
2021-05-14T00:52:34.68+00:00

I have a number of AppServices running with health-checks turned on. I recently created a new one based on the same template, and for some reason its health checks seem to constantly spike, running a full order of magnitude more frequently than the other.

As an example, in the last hour, each of my other AppServices have received approximately 600 health check requests (there are 10 instances running).

The AppService in question has received 2700 health check requests in that same amount of time. It's on the same AppService Plan.

This can also be visualized pretty easily by looking at the requests graph for the App Service.

Good:
96564-image.png

Bad:
96563-image.png

Is there some setting I can tweak to bring this request load down? It seems pretty absurd to check the health of a service 3000 times an hour.

Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
7,149 questions
0 comments
Accepted answer
  1. Matt Robold (WCSG) 96 Reputation points
    2021-05-18T22:35:39.59+00:00

    Hi Grace,

    It was still happening, but your answer was very helpful. I had been checking logs for failed requests and not seeing any, but I changed my AppInsights query to look for any request that wasn't 200 and found that I was getting a deluge of 307s.

    It turns out the AppService's TLS settings were incorrect - they were not set for HTTPS-only so the app was attempting health checks over normal HTTP. Changed HTTPS-Only to ON and the requests dropped after a few minutes.

    Thank you.

    MR

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Grmacjon-MSFT 17,136 Reputation points
    2021-05-18T08:50:02.85+00:00

    Hi @Matt Robold (WCSG) ,

    Thanks for bringing this to our attention this is definitely odd behavior. Are you still experiencing this issue? What was the response status code for the AppService in question? Did you get ping failures from any of your instances?

    Usually, if an instance doesn't respond with a status code between 200-299 (inclusive) after two or more requests, or fails to respond to the ping, the system determines it's unhealthy and removes it. After removal, Health check continues to ping the unhealthy instance. If it continues to respond unsuccessfully, App Service restarts the underlying VM in an effort to return the instance to a healthy state.
    If an instance remains unhealthy for one hour, it will be replaced with new instance.

    Here are some app settings configurations you can add:
    97349-healthcheck.png

    Let us know if you have further questions.

    Best,
    -Grace

    0 comments