CosmosDb change feed duplicate data read issue

Raj Kumar Dey 1 Reputation point
2021-06-17T18:23:19.473+00:00

I am working with Azure function where cosmosdb trigger is being used to track changes against an orderId. But i am seeing azure function is picking the same changeid(logging the changeid coming from changefeed in azure function) for the same order. It is creating duplicate data in downstream.

Can anyone help us to sort this out.

The reason for invoking the same changeid for the same orderId

in below link i am seeing there might be duplicate read for "Eventual consistency" level.

Consistency level

While consuming the change feed in an Eventual consistency level, there could be duplicate events in-between subsequent change feed read operations (the last event of one read operation appears as the first of the next).

https://learn.microsoft.com/en-us/azure/cosmos-db/change-feed

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,911 questions
Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,902 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. KalyanChanumolu-MSFT 8,351 Reputation points
    2021-06-18T14:11:20.203+00:00

    @Raj Kumar Dey Welcome to Microsoft Q&A forums and thank you for reaching out.

    The Cosmos DB Trigger has an "at least once" delivery, which means that an item could be, in some cases, delivered more than once.
    The same document could be processed again if the number of instances is changing due to a scaling event, where the Trigger will rebalance the leases across new instances.
    We have documented guidance on the troubleshooting page - Some changes are repeated in my Trigger

    ----------

    If an answer is helpful, please "Accept answer" or "Up-Vote" which might help other community members reading this thread.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.