Cosmos DB archiving mechamism

Eliot Chen 86 Reputation points
2022-01-28T03:01:02.183+00:00

What is the best way to archive records (JSON) older than X months to a cold blob storage storage? Is ADF the answer or there is a less involved/simple way to achieve this?

https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-cosmos-db?tabs=data-factory

Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,436 questions
{count} vote

Accepted answer
  1. Anurag Sharma 17,566 Reputation points
    2022-01-28T05:43:35.247+00:00

    Hi @Eliot Chen , welcome to Microsoft Q&A forum.

    Azure Data Factory is definitely one of the most commonly used method to migrate/archive data from Azure Cosmos DB. However there are few other ways also to achieve this:

    1) Using Azure Cosmos Change Feed: Any new document that gets added to Azure Cosmos db account can be inserted into another storage , however this is not available out of box and we need to write this functionality using azure Function or any other way. This also comes with caveat that change feed does not support 'Delete' currently. This could increase the cost as we are maintaining 2 copies of data simultaneously.

    Azure Cosmos DB - Change Feed Lab

    2) Azure Cosmos DB Data Migration Tool: Using this we can firstly select the source Azure Cosmos DB account and then write specific query to select the records we want to archive like based on data or any other attribute. After that we can select the target and save the data in JSON format to either local file or Blob.

    Source:

    169295-image.png

    Target:

    169296-image.png

    Data migration tool to migrate your data to Azure Cosmos DB

    Advantage of using ADF or Azure function is there is no manual intervention here. But if this archival process does not happen too frequently and can be manual then we can choose to use Data Migration Tool as well.

    Please let us know if this helps or else we can discuss further on the same. If answer helps, you can mark it 'Accept Answer'.

    1 person found this answer helpful.
    0 comments No comments

1 additional answer

Sort by: Most helpful
  1. Eliot Chen 86 Reputation points
    2022-01-28T05:51:55.137+00:00

    @AnuragSharma-MSFT, thanks for your feedback!