Share via

Seek architectural advice: Incoming event/message batching for multiple partitions/sessions with Functions.

Plamen Petrow 20 Reputation points
2025-05-30T19:51:54.4933333+00:00

Hi community and support,

following scenario:
We want to receive messages (load > 1 event per second) and batch them for further processing.
The messages have a "sorting" property (let's call it partition), and we only want to batch messages having the same partition value.
After receiving a first message with a specific partition value, we want to wait some time for follow-up messages with the same partition value. The waiting time should be reset on each incoming message, as long as messages with the partition value are coming in, or up to a maximal overall time.

We would like to use Functions in the isolated-process model (avoiding the in-process model), preferably with session enabled ServiceBus topics (but open to other suggestions) with partition value = session id.

Can this be done and how?
We are looking for a way to receive the messages with a correspondence of the ServiceBusReceiveActions (which seems to be missing for the isolated-process model) or a way to get the ServiceBusReceiver for the current session.

Thanks!

Azure Functions
Azure Functions

An Azure service that provides an event-driven serverless compute platform.


Answer accepted by question author

Gaurav Kumar 790 Reputation points Moderator
2025-06-05T10:38:26.7333333+00:00

Hi Plamen Petrow,

Thanks for your detailed scenario and for trying the session-based batching with Azure Functions (Isolated Worker).

Issue:

The issue you're encountering (SessionCannotBeLocked) is expected. When you use [ServiceBusTrigger(..., IsSessionsEnabled = true)], Azure Functions internally locks the session before your function code runs. This means any call to AcceptSessionAsync on the same session will fail with a session lock conflict, as you're seeing.

Solution:

Use a Background Worker Instead of ServiceBusTrigger

To implement custom batching logic (e.g., buffer messages for the same session with idle and max timeout conditions), it's best not to use ServiceBusTrigger at all. Instead, use a hosted background service that manually handles session locks and message batching.

Create a hosted service in your .NET isolated process (within the same Azure Function App or another .NET app), and manually control session receivers.

public class SessionBatchWorker : BackgroundService
{
    private readonly ServiceBusClient _client;
    public SessionBatchWorker(ServiceBusClient client)
    {
        _client = client;
    }
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                ServiceBusSessionReceiver receiver = await _client.AcceptNextSessionAsync("my-session-queue", cancellationToken: stoppingToken);
                _ = ProcessSessionAsync(receiver, stoppingToken);
            }

            catch (ServiceBusException ex) when (ex.Reason == ServiceBusFailureReason.ServiceTimeout)

            {

                await Task.Delay(1000, stoppingToken);
            }
        }
    }
    private async Task ProcessSessionAsync(ServiceBusSessionReceiver receiver, CancellationToken cancellationToken)
    {
        var messages = new List<ServiceBusReceivedMessage>();
        var idleTimeout = TimeSpan.FromSeconds(5);
        var overallTimeout = TimeSpan.FromSeconds(30);
        var lastReceived = DateTime.UtcNow;
        while (DateTime.UtcNow - lastReceived < idleTimeout && DateTime.UtcNow - lastReceived < overallTimeout)

        {

            var message = await receiver.ReceiveMessageAsync(TimeSpan.FromSeconds(1), cancellationToken);

            if (message != null)
            {
                messages.Add(message);
                lastReceived = DateTime.UtcNow;
            }
        }

        foreach (var msg in messages)

        {

            Console.WriteLine($"Processing {msg.Body}");

            await receiver.CompleteMessageAsync(msg, cancellationToken);
        }
        await receiver.CloseAsync();
    }
}

Then just register it in Program.cs:


builder.Services.AddSingleton<ServiceBusClient>(sp => new ServiceBusClient("<your-connection-string>"));

builder.Services.AddHostedService<SessionBatchWorker>();

Please keep this in mind:

  • This approach bypasses the function trigger binding entirely, you're controlling the receiver yourself, allowing batching, timeouts, etc.
  • Session handling is exclusive, your worker will be safe per session.
  • This will scale better if you allow multiple concurrent session workers (limit with MaxConcurrentSessions).
  • Use queue or topic subscription with RequiresSession = true.

For more details, please refer the following Microsoft documentation:
Service Bus Sessions
Azure Functions Service Bus Trigger
ServiceBusSessionReceiver
Azure Functions Isolated Worker


I hope this information helps.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Was this answer helpful?

1 person found this answer helpful.

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.