How does Cosmos TTL works, I want to purge the items which are older than 365 days

Question

How does Cosmos TTL works, I want to purge the items which are older than 365 days

Nikhil Kuttan 5

I want to purge the items which are older than 365 days, If I set 31536000 sec
will it flush all the items or it will flush only those items which were not modified in last 365 days

2 answers

Your answer

Answer 1

Vahid Ghafarpour 23,385 Volunteer Moderator

In Azure Cosmos DB, the Time-to-Live (TTL) feature works based on the item's last modification time, not the creation time. The TTL value specifies the duration in seconds after the last update to an item, after which the item is eligible for deletion.

When an item is created or updated in Cosmos DB, the TTL value is reset. If the TTL value is set to a specific duration, such as 365 days (31536000 seconds), and no further updates are made to the item within that duration, the item will be marked for deletion.

Nikhil Kuttan 5 Reputation points

2023-05-31T17:38:22.4366667+00:00

Thanks for your reply
What if a new item comes in will it remain in the container.
basically, I want to only keep last 365 days data
Vahid Ghafarpour 23,385 Reputation points Volunteer Moderator

2023-05-31T17:43:18.94+00:00

Cosmos DB periodically scans the items and checks their TTL values. When an item's TTL value expires, it is marked for deletion.

Cosmos DB automatically performs background deletion of expired items. The actual deletion process is asynchronous, so it may take some time before the expired items are fully purged from the system. The frequency of the background process depends on various factors, such as container throughput and the volume of expired items.

It's important to note that TTL is not guaranteed to delete items immediately after the expiration time. The background deletion process operates in a probabilistic manner and can take some time to complete, especially if there are a large number of expired items or high container throughput.

Using TTL for item expiration is a convenient way to manage data retention in Cosmos DB automatically. However, if you require precise control over when and how items are purged, or if you need to perform custom logic based on item age, you may need to implement your own deletion process, as described in the previous answer.
Vahid Ghafarpour 23,385 Reputation points Volunteer Moderator

2023-05-31T17:44:27.1833333+00:00

Cosmos DB periodically scans the items and checks their TTL values. When an item's TTL value expires, it is marked for deletion.

Cosmos DB automatically performs background deletion of expired items. The actual deletion process is asynchronous, so it may take some time before the expired items are fully purged from the system. The frequency of the background process depends on various factors, such as container throughput and the volume of expired items.

It's important to note that TTL is not guaranteed to delete items immediately after the expiration time. The background deletion process operates in a probabilistic manner and can take some time to complete, especially if there are a large number of expired items or high container throughput.

Using TTL for item expiration is a convenient way to manage data retention in Cosmos DB automatically. However, if you require precise control over when and how items are purged, or if you need to perform custom logic based on item age, you may need to implement your own deletion process, as described in the previous answer.

Answer 2

@Nikhil Kuttan Thank you for reaching out.

What if a new item comes in will it remain in the container.
basically, I want to only keep last 365 days data.

In addition to the above answer.

If a new item is added to the container, it will be subject to the same TTL policy as the existing items. If the new item is not modified within the last 365 days, it will be automatically removed from the container when the TTL expires. If the new item is modified within the last 365 days, it will remain in the container until the TTL expires.

To only keep the last 365 days of data in the container, you can set the TTL value to 31536000 seconds (which is equivalent to 365 days). This will automatically remove all items that were last modified more than 365 days ago. Any new items that are added to the container will also be subject to the same TTL policy and will be automatically removed when the TTL expires.

Time to Live (TTL) in Azure Cosmos DB

Regards,

Oury

Oury Ba-MSFT 20,936 Reputation points Microsoft Employee Moderator

2023-06-01T03:26:50.9533333+00:00

@Nikhil Kuttan Please don't forget to mark as accept answer wherever information provided were helpful.

Regards,

Oury

Share via

How does Cosmos TTL works, I want to purge the items which are older than 365 days

2 answers

Your answer