How to delete documents from analytical storage?

Question

Looking for clarity on the ATTL value for Cosmos DB containers with analytical storage enabled. If I'm reading the documentation correctly, data never gets deleted from analytical storage? You can always just set the ATTL to -1, and every single document that was replicated over to analytical storage for the container's lifetime will be queryable from a Synapse product/workflow?

Continuing, if you set the ATTL to 0, will no new documents be replicated to analytical storage so long as it's set to 0? Analytical storage is read-only, meaning we can't physically delete these entries ourselves; we'd have to delete the data container, which will then delete the analytical store?

And if we delete the document from our data container before it expires, it will still exist in analytical storage?

Thinking about this from a PII/data compliance perspective...If we have TTLs on our actual data container to meet compliance, it seems like analytical storage is not a route we want to take since its ATTL value is really just a sliding window to hide the data, but that window can be moved back to reveal it whenever we want?

Answer 1

Hi Murillo, Antonio J •,

Welcome to Microsoft Q&A forum.

As I understand, you want to delete documents from analytical storage of Azure Cosmos DB.

The retention of this transactional data in analytical store can be controlled at container level by the AnalyticalStoreTimeToLiveInSeconds property.

Currently we don't delete any data from analytical store. If you set your ATTL to any positive integer, the data won't be included in your queries and you won't be billed for it. But if you change ATTL back to -1, all the data will show up again, you will start to be billed for all the data volume.

Can I update or delete an item from the analytical store after it has been TTL’d out from the transactional store?

All transactional updates and deletes are copied to the analytical store but if the item has been purged from the transactional store, then it can't be updated in the analytical store. To learn more, see the Analytical TTL article.

Hope this helps. Let us know if you have a different ask.

Thanks