Change feed modes in Azure Cosmos DB
APPLIES TO:
NoSQL
Azure Cosmos DB offers two change feed modes. Each mode offers the same core functionality. Differences include the operations that are captured in the feed, the metadata that's available for each change, and the retention period of changes. You can consume the change feed in different modes across multiple applications for the same Azure Cosmos DB container to fit the requirements of each workload. Each individual change feed application can only be configured to read change feed in one mode. Consuming the change feed in one mode doesn't prohibit you from consuming the change feed in another mode in a different application.
Note
Do you have any feedback about change feed modes? We want to hear it! Feel free to share feedback directly with the Azure Cosmos DB engineering team: cosmoschangefeed@microsoft.com.
Latest version change feed mode
Latest version mode is a persistent record of changes made to items from creates and updates. You get the latest version of each item in the container. For example, if an item is created and then updated before you read the change feed, only the updated version appears in the change feed. Deletes aren't captured as changes, and when an item is deleted, it's no longer be available in the feed. Latest version change feed mode is enabled by default and is compatible with all Azure Cosmos DB accounts except the API for Table and the API for PostgreSQL. This mode was previously the default way to consume the change feed.
All versions and deletes change feed mode (preview)
All versions and deletes mode (preview) is a persistent record of all changes to items from create, update, and delete operations. You get a record of each change to items in the order that it occurred, including intermediate changes to an item between change feed reads. For example, if an item is created and then updated before you read the change feed, both the create and the update versions of the item appear in the change feed. To read from the change feed in all versions and deletes mode, you must have continuous backups configured for your Azure Cosmos DB account. Turning on continuous backups creates the all versions and deletes change feed. You can only read changes that occurred within the continuous backup period when using this change feed mode. This mode is only compatible with Azure Cosmos DB for NoSQL accounts. Learn more about how to sign up for the preview.
Change feed use cases
Latest version mode provides an easy way to process both real-time and historic changes to items in a container with the ability to go back to changes from the beginning of the container.
The following are scenarios well-suited to this mode:
Migrations of an entire container to a secondary location.
Ability to reprocess changes from the beginning of the container.
Real-time processing of changes to items in a container resulting from create and update operations.
Workloads that don't need to capture deletes or intermediate changes between reads.
Features of each mode
In addition to the common features across all change feed modes, each change feed mode has the following characteristics:
The change feed includes insert and update operations that are made to items in the container.
This mode of change feed doesn't log deletes. You can capture deletes by setting a "soft-delete" flag within your items instead of deleting them directly. For example, you can add an attribute in the item called
deleted
with the valuetrue
, and then set a Time to Live (TTL) on the item. The change feed captures it as an update and the item is automatically deleted when the TTL expires. Alternatively, you can set a finite expiration period for your items by using the TTL capability. With this solution, you have to process the changes within a shorter time interval than the TTL expiration period.Only the most recent change for a specific item is included in the change feed. Intermediate changes might not be available.
When an item is deleted, it's no longer available in the change feed.
Changes can be synchronized from any point in time, and there's no fixed data retention period for which changes are available.
You can't filter the change feed for a specific type of operation. One possible alternative, is to add a "soft marker" on the item for updates and filter based on the marker when you process items in the change feed.
The starting point to read change feed can be from the beginning of the container, from a point in time, from "now," or from a specific checkpoint. The precision of the start time is approximately five seconds.
Work with the change feed
Each mode is compatible with different methods to read the change feed for each language.
You can use the following ways to consume changes from change feed in latest version mode:
Method to read change feed | .NET | Java | Python | Node.js |
---|---|---|---|---|
Change feed pull model | Yes | Yes | Yes | Yes |
Change feed processor | Yes | Yes | No | No |
Azure Functions trigger | Yes | Yes | Yes | Yes |
Parse the response object
In latest version mode, the default response object is an array of items that have changed. Each item contains the standard metadata for any Azure Cosmos DB item, including _etag
and _ts
, with the addition of a new property, _lsn
.
The _etag
format is internal and you shouldn't take dependency on it because it can change anytime. _ts
is a modification or a creation time stamp. You can use _ts
for chronological comparison. _lsn
is a batch ID that is added for change feed only that represents the transaction ID. Many items can have same _lsn
.
ETag
on FeedResponse
is different from the _etag
you see on the item. _etag
is an internal identifier, and it's used for concurrency control. The _etag
property represents the version of the item, whereas the ETag
property is used to sequence the feed.
Next steps
Learn more about change feed in the following articles: