Function App, multithreading optimization

Question

I have multiple IoT devices that send telemetry data to Azure IoT Hub, which then triggers my Azure Function for processing. The processed sensor data is stored in Azure Table Storage, with each device having its own table.

I would like to improve data retrieval through the Function App, as it is currently slow because I am reading multiple partitions sequentially rather than in parallel. My Function App is on a consumption plan.

List<string> dailyPKs = BuildDailyPartitionKeysUtc(starttime, endtime);                 
    var bag = new System.Collections.Concurrent.ConcurrentBag<AzureTableSensorPlotData>();
    using var sem = new System.Threading.SemaphoreSlim(8); // tune concurrency (4–16 is typical)

    var tasks = dailyPKs.Select(async pk =>
    {
        await sem.WaitAsync().ConfigureAwait(false);
        try
        {                                
            string Start_ticks = (DateTime.MaxValue.Ticks - starttime.Ticks).ToString("D22");
            string End_ticks = (DateTime.MaxValue.Ticks - endtime.Ticks).ToString("D22");

            var rows = await AzureTable.Read_Azure_IoT_Device_For_App_Graph_View(
                average_data: average_data,
                tableid: "AzureDevice" + device_plotb,
                partitionKey_start: pk,
                partitionKey_end: pk,
                rowKey_start: End_ticks,
                rowKey_end: Start_ticks, 
                id: id,
                log: log,
                partionquerymode:1
            ).ConfigureAwait(false);

            if (rows != null)
                foreach (var r in rows) bag.Add(r);
        }
        finally
        {
            sem.Release();
        }
    });
    await Task.WhenAll(tasks).ConfigureAwait(false);
    var result = bag.OrderByDescending(d => d.Timestamp);
    jsonToReturn = JsonConvert.SerializeObject(result);

I want to ensure I’m fully utilising the available memory and CPU resources within a single instance, without exceeding limits or causing exceptions due to excessive threading.

My question

Given that Azure Functions manage scaling automatically, how can I safely optimise the number of concurrent threads (beyond eight) to maximise throughput? Or would I be better in calling maximum as possible and placing them in a durable function? With the Durable function, i have noticed it can be, at times have a little delay

Answer 1

Hi @DJ , Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

1. Instrumentation
   • Turn on Application Insights, and enable Storage metrics including ThrottledRequests. Collect CPU/memory, response times, dependency calls, and exceptions.
2. Code fixes
   • Make TableClient or storage client static and reused.
   • Add bounded SemaphoreSlim (value determined from the test above).
   • Add robust retry logic (exponential backoff) for transient 429s and 5xx. Use SDK retry policies not homemade ones when possible.
3. If you need more throughput than a single instance can provide
   • Option A — Queue fan-out: push per-partition work into a queue and let many functions instances process in parallel (no single instance overload). Good for high throughput and elastic scale.
   • Option B — Durable Functions fan-out/fan-in: orchestrator calls activities for each partition, Durable manages scaling/retries. Use if you want checkpointing and easier orchestration but accept small latency overhead.
   • Option C — Upgrade to Premium: more CPU/memory per instance and predictable scaling if you need heavier per-instance concurrency.
4. Data model & Storage changes
   • If partitions are uneven (hot partitions), redesign partitioning to distribute load. Consider multiple storage accounts if a single account is the bottleneck.

If still facing issue, please share below information in Private messages.

• Function runtime version and language (dotnet version / Functions runtime v3/v4/v4.x).
• Which Table SDK version (v2 vs v12) and whether clients are being reused.
• Average rows per partition, and average row size (KB).
• Observed App Insights/Function metrics: CPU% and memory at current runs, and any 429 or socket errors.
• Whether low latency per request is more important than throughput (latency-sensitive vs throughput batch).
• Whether moving to Premium or adding queues/Durable Functions is acceptable architecturally and cost-wise.

Answer 2

Hello DJ

Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

1.Dynamic vs static SemaphoreSlim value: The “Concurrency in Azure Functions” doc and the host.json reference show that you can enable dynamic concurrency rather than a hard-coded fixed maximum. (See host.json concurrency.dynamicConcurrencyEnabled setting)

https://learn.microsoft.com/en-us/azure/azure-functions/functions-host-json

https://learn.microsoft.com/en-us/azure/azure-functions/functions-concurrency

2.queue fan-out pattern: The “Queue trigger for Azure Functions” doc shows how message batches are processed, and the “Durable Functions” docs show how fan-out/fan-in is implemented (though you can achieve fan-out without Durable by using storage queue messages)

Azure Queue storage trigger for Azure Functions

Durable Functions

Fan-out/fan-in scenario in Durable Functions

3.limits per invocation / threads / memory: The “Scale and hosting” and “Event-driven scaling” docs indicate each instance in a Consumption plan is typically limited to ~1 CPU/core + ~1.5 GB memory. Event-driven scaling in Azure Functions

Azure Functions hosting options

Event-driven scaling in Azure Functions

I hope this helps in resolving the issue, do let me know if you have any further questions on this.

Thanks