Edit

Share via

Migrate data to Azure Cosmos DB using the desktop data migration tool

  • Article

APPLIES TO: NoSQL MongoDB Table

The Azure Cosmos DB desktop data migration tool is an open-source command-line application to import or export data from Azure Cosmos DB. The tool can migrate data to and from many sources and sinks including, but not limited to:

  • Azure Cosmos DB for NoSQL
  • Azure Cosmos DB for MongoDB
  • Azure Cosmos DB for Table
  • Azure Table storage
  • JSON
  • MongoDB
  • SQL Server

Important

For this guide, you will perform a data migration from JSON to Azure Cosmos DB for NoSQL.

Prerequisites

Install the desktop data migration tool

First, install the latest version of the desktop data migration tool from the GitHub repository.

Note

The desktop data migration tool requires .NET 6.0 or later on your local machine.

  1. In your browser, navigate to the Releases section of the repository: azurecosmosdb/data-migration-desktop-tool/releases.

  2. Download the latest compressed folder for your platform. There are compressed folders for the win-x64, mac-x64, and linux-x64 platforms.

  3. Extract the files to an install location on your local machine.

  4. (Optional) Add the desktop data migration tool to the PATH environment variable of your local machine.

Prepare your migration target

Next, create a target database and container on the Azure Cosmos DB for NoSQL account.

  1. Open a new terminal. If you haven't already, sign in to the Azure CLI.

  2. Create new shell variables for the Azure Cosmos DB account's name and resource group.

    Azure CLI 
    # Variable for Azure Cosmos DB account name
accountName="<name-of-existing-account>"

# Variable for resource group name
resourceGroupName="<name-of-existing-resource-group>"

  3. Create a new database using az cosmosdb sql database create. Name the new database cosmicworks and configure the database with 400 RU/s of shared throughput.

    Azure CLI 
    az cosmosdb sql database create \
    --resource-group $resourceGroupName \
    --account-name $accountName \
    --name cosmicworks \
    --throughput 400

  4. Use az cosmosdb sql container create to create a new container named products within the cosmicworks database. Set the partition key path of the new container to /category.

    Azure CLI 
    az cosmosdb sql container create \
    --resource-group $resourceGroupName \
    --account-name $accountName \
    --database-name cosmicworks \
    --name products \
    --partition-key-path "/category"

  5. Find the primary connection string from the list of keys for the account with az cosmosdb keys list.

    Azure CLI 
    az cosmosdb keys list \
    --resource-group $resourceGroupName \
    --name $accountName \
    --type connection-strings

  6. Record the primary connection string value. You use this credential later when migrating data with the tool.

Perform a migration operation

Now, migrate data from a JSON array to the newly created Azure Cosmos DB for NoSQL container.

  1. Navigate to an empty directory on your local machine. Within that directory, create a new file named migrationsettings.json.

  2. Within the JSON file, create a new empty JSON object:

    JSON 
    {}

  3. Create a new property named Source with the value json. Create another new property named SourceSettings with an empty object as the value.

    JSON 
    {
  "Source": "json",
  "SourceSettings": {}
}

  4. Within the SourceSettings object, create a new property named FilePath with the value set to this URI: https://raw.githubusercontent.com/azure-samples/cosmos-db-migration-sample-data/main/nosql-data.json.

    JSON 
    {
  "Source": "json",
  "SourceSettings": {
    "FilePath": "https://raw.githubusercontent.com/azure-samples/cosmos-db-migration-sample-data/main/nosql-data.json"
  }
}

  5. Create another new property named Sink with the value cosmos-nosql. Also, create a property named SinkSettings with an empty object.

    JSON 
    {
  "Source": "json",
  "SourceSettings": {
    "FilePath": "https://raw.githubusercontent.com/azure-samples/cosmos-db-migration-sample-data/main/nosql-data.json"
  },
  "Sink": "cosmos-nosql",
  "SinkSettings": {
  }
}

  6. Within SinkSettings, create a property named ConnectionString with the primary connection string you recorded earlier in this guide as its value.

    JSON 
    {
  "Source": "json",
  "SourceSettings": {
    "FilePath": "https://raw.githubusercontent.com/azure-samples/cosmos-db-migration-sample-data/main/nosql-data.json"
  },
  "Sink": "cosmos-nosql",
  "SinkSettings": {
    "ConnectionString": "<connection-string-for-existing-account>"
  }
}

  7. Add Database, Container, and PartitionKeyPath properties with cosmicworks, products, and /category as their values respectively.

    JSON 
    {
  "Source": "json",
  "SourceSettings": {
    "FilePath": "https://raw.githubusercontent.com/azure-samples/cosmos-db-migration-sample-data/main/nosql-data.json"
  },
  "Sink": "cosmos-nosql",
  "SinkSettings": {
    "ConnectionString": "<connection-string-for-existing-account>",
    "Database": "cosmicworks",
    "Container": "products",
    "PartitionKeyPath": "/category"
  }
}

  8. Save the migrationsettings.json file.

  9. Open a new terminal and navigate to the directory containing your migrationsettings.json file.

  10. Run the desktop data migration tool using the dmt command.

    terminal 
    dmt

    Note

    If you did not add the installation path to your PATH environment variable, you may need to specify the full path to the dmt executable.

  11. The tool now outputs the sources and sinks used by the migration.

    Output 
    Using JSON Source
Using Cosmos-nosql Sink