Delayed Database Calls on Azure Function App

Question

I have multiple Azure Function apps running on a dedicated app service plan. Recently I have been getting slow responses from all of these. The problem is that the response are not slow continuously, but intermittently.

For example, a HTTP triggered function is called four times with same data. The function returns response within a second three times, but returns response after 20 or 40 seconds once. I did not find any pattern on the duration of delay. I have checked CPU, memory and outbound TCP connection metrics. but did not find anything that can indicate the apps need more resource. Azure SQL metrics also show everything to be fine.

Only issue I can find is that the traces show async and sync database call have a delay before actually starting a database call. This delay is not the same for a single API request either. That is if a function HTTP API request has multiple database calls, only one or two has a delay, but others are fine.

Does anyone have any idea why this could be happening or how I can debug this? Dev environment runs have no such issue.

Update: I ran the application insights profiler on one of the Azure Function App and it looks like the majority of time is being spent in the following location when response time is high. But app service or function app metrics do not show any SNAT exhaustion issue. Then what is going on here?

Answer 1

Good day Asiful Nobel,

I am pretty sure (guessing) that the issue is related to the Azure function and not the database, but from the database side here are some points which might (low chance) provide some insights

(1) In general any connection has latency of the connection itself (for example the authentication procedure)

(2) Azure SQL database has option to use Serverless tier. In this case, Compute resources are auto-scaled​. The chance that this is your issue is extremely low but scaling has latency and in theory if your function lead to scaling of the database then you might get delay.

(3) Azure SQL database like any other transaction based database, can have locked and waits - In very very simple words, this mean that one query need to wait to another query which uses the same resources (for example two queries which try to update the same row at the same time).

My guess is that you need to focus on the Azure function

Note: If the issue related to the DATABASE then we will need a lot more information and to monitor the database