AKS cold start?

Deyan Petrov 1 Reputation point
2020-06-15T12:14:31.533+00:00

I have about 30 pods (F#/.NET Core + Azure Functions Runtime) running in a 3-node AKS cluster, each of which I am calling every 5 minutes externally (using Application Insights Availability/Web tests).
I am observing that if the pod is not called often (e.g. only 1x/every 5 minutes) then it takes 5000ms+ to call it (usually the call checks dependencies like if db is reachable, if storage blob container is reachable, or event hub). If I call the pod 10-20 times one after the other, I get very fast reponse times below 100ms.

First I thought the shared storage account (AzureWebJobsStorage env var) could be the issue, but then took 1 pod and reassigned it to a dedicated storage account, and after couple of minutes of inactivity again 5000ms+ response time.

Is there any kind of cold start, or automatic shut down of the pods in case they are inactive for a couple of minutes??

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,607 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,977 questions
0 comments No comments
{count} votes

4 answers

Sort by: Most helpful
  1. Deyan Petrov 1 Reputation point
    2020-06-15T12:45:05.747+00:00

    I have increased the number of calls to 1 pod and it seems that I do not experience such "cold-starts" with this pod now ... Somehow it seems there is some "idle timeout" and calling the pod 1x/every 5 minutes is too less to keep it alive - can this be??

    0 comments No comments

  2. Deyan Petrov 1 Reputation point
    2020-06-15T13:38:59.387+00:00

    Trying now with "functionTimeout": "-1" in host.json ....


  3. Deyan Petrov 1 Reputation point
    2020-06-15T17:45:34.233+00:00

    I have found numerous posts about "5 seconds dns resolve kubernetes" - can this be the reason?

    E.g.

    0 comments No comments

  4. Deyan Petrov 1 Reputation point
    2020-06-16T06:17:40.527+00:00

    10147-dnsfix.png

    Applied an old workaround I found googling (under spec/template/spec, same level as containers), seems to help ... Wondering why 1,5 year-old-problem is hitting on innocent AKS/K8s newbies like me in 2020 :(

      dnsConfig:  
        options:  
        - name: single-request-reopen  
    
    0 comments No comments