Optimistic Concurrency is not working in Azure Table Storage

Question

Hi,
I have a Service Bus Topic Trigger function that processes each message and updates the count of each message processed in Table Storage. When the count of entities in Table Storage becomes equal to the TotalCount of each message, I send a message to another Topic.
MessageBody: {"ExecutionId":"c9e7266c-a632-4a10-9079-55466e832c3f","TotalCount":5,"Data":"Id":1234}}

This is the method I use to update the count in Table Storage:

private async Task<int?> UpdateExecutionCountAsync(string executionId, int totalCount, bool isDeadLetterMessage)
{
    var retryPolicy = Policy
        .Handle<RequestFailedException>(ex => ex.Status == 409 || ex.Status == 412)
        .WaitAndRetryAsync(5, retryAttempt => TimeSpan.FromSeconds(2));

    return await retryPolicy.ExecuteAsync(async () =>
    {
        try
        {
            var existingEntity = await _tableStorageService.GetEntityIfExistsAsync(executionId, executionId);
            var executionCount = existingEntity?.Count ?? 0;
            var eTag = existingEntity?.ETag;

            if (existingEntity == null)
            {
                var entity = new FunctionExecutionEntity
                {
                    PartitionKey = executionId,
                    RowKey = executionId,
                    Count = 1
                };

                await _tableStorageService.AddEntityAsync(entity);
                executionCount = 1;
                _logger.LogInformation($"New entity created for executionId: {executionId} with count {executionCount}");
            }
            else if (totalCount != existingEntity.Count)
            {
                existingEntity.Count++;
                await _tableStorageService.UpdateEntityAsync(entity, eTag);
                executionCount = existingEntity.Count;
                _logger.LogInformation($" Existing entity updated for executionId: {executionId} with count {executionCount}");
            }
            else
            {
                _logger.LogInformation($"Entity already updated by another instance");
            }

            return executionCount;
        }
        catch (RequestFailedException ex)
        {
            _logger.LogError($"UpdateExecutionCountAsync: Error {ex.Status}: {ex.Message}");
            throw;
        }
    });
}

I have a retry policy in place for handling 412 or 409 status codes. Despite this, the TotalCount is not being updated in Table Storage.

Answer 1

Hi Sonal,

Please check this blog if this helps.

https://itgigwiz.com/2024/01/04/ensuring-consistent-updates-with-optimistic-concurrency-in-azure-table-storage/

Do let us know if it didn't help. i would happy to assist you.

BR

Vasim

Answer 2

Hello sonal khatri,

Greetings! Welcome to Microsoft Q&A forum.

Optimistic concurrency is a strategy used to handle concurrent updates to data, ensuring that changes don’t overwrite each other unintentionally.

there are some possible ways to ensure that the concurrency is working correctly, Optimistic concurrency in Azure Table Storage is handled through ETag property on the entity. Anytime an entity is updated, its ETag value changes.

The process of updating an entity using optimistic concurrency is something like the following:

1. You fetch the entity from the table.
2. You make changes to the entity on the client side (say increase the version property).
3. You send the update request to Table Storage. When sending the update request, you will need to include the ETag value of the fetched entity.

When ETag value is included in the update request, Table Storage compares that value with the current ETag value of the entity.

If both are the same, that means the entity has not been updated since fetched and updates can be done.

If the values are different, then Table Storage returns a Pre Condition failed (412) error back. In this case, you will need to fetch the entity again and repeat the process.

refer remarks section - https://learn.microsoft.com/en-us/rest/api/storageservices/update-entity2#remarks for detailed guidance on etag.

refer - https://azure.microsoft.com/en-us/blog/managing-concurrency-in-microsoft-azure-storage-2/

Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.