How to do a bulk upload of 17.6M records in cosmos db for mongo api instance in less than an hour?

Question

Currently I am using a serverless instance of Cosmos db for mongo api in Canada East region. It takes almost a day to write 17.6M records to the cosmos instance. I have enabled the server side retries and I am using only a single instance of mongoClient throughout my application. The bulk executor library is currently not supported for Cosmos db for mongo api version. Are there any other ways to write these much data faster?

Note: I only want to write these much data once in the lifetime of the db instance. So going for a higher throughput will not be an ideal solution as I wont be needing it always.

Answer 1

Hi 15643986 ,

Welcome to Microsoft Q&A forum and thanks for using Azure services.

As I understand, you want to perform bulk upload in Azure Cosmos MongoDB API.

Since you have explored many options, I would recommend you to use mongoimport if helps in your usecase.

Check here: Tutorial: Migrate MongoDB to Azure Cosmos DB's API for MongoDB offline using MongoDB native tools

Additionally, you can also refer to:

• Migrate data from MongoDB to an Azure Cosmos DB for MongoDB account by using Azure Databricks
• Tutorial: Migrate MongoDB to Azure Cosmos DB for MongoDB offline

Hope above information would help. If not, please let us know so that we can further assist you.

Thank you.