Edit

Share via

Azure Cosmos DB for MongoDB migration extension for Azure Data Studio (preview)

  • Article

The Azure Cosmos DB for MongoDB migration extension helps you in migrating your MongoDB workloads to Azure Cosmos DB. You can use this extension to:

  • Run an end-to-end assessment on your workload and find out the necessary actions you need to take to migrate your workloads to vCore-based Azure Cosmos DB for MongoDB.
  • Perform the migration operation with selected databases and collections to vCore-based Azure Cosmos DB for MongoDB.

Important

Currently this extension doesn't support the following scenarios:

  • Migration where source or target account are Private Endpoint enabled.
  • Online/Offline Migrations for RU-based Azure Cosmos DB for MongoDB.

For more information on alternate solutions for the API for MongoDB vCore, see migration options.

Prerequisites

Prepare

Before starting the migration, carry out up-front planning and decision-making about your migration before you actually move any data.

Register Microsoft.DataMigration resource provider in your subscription

To ensure that the Microsoft.DataMigration resource provider is registered in your subscription, you can follow these steps:

Azure portal:

  1. Go to the Azure portal and navigate to your subscription.
  2. In the left-hand menu, select Resource providers under Settings.
  3. Search for Microsoft.DataMigration in the search box at the top.
  4. If it is not registered, select it and click on the Register button.

Azure CLI:

  1. Open the Azure Cloud Shell or your local terminal.
  2. Run the following command to register the resource provider:
az provider register --namespace Microsoft.DataMigration

PowerShell:

  1. Open the Azure Cloud Shell or your local PowerShell.
  2. Run the following command to register the resource provider:
Register-AzResourceProvider -ProviderNamespace "Microsoft.DataMigration"

Install the extension

Install the Azure Cosmos DB for MongoDB migration extension in Azure Data Studio before you begin your migration.

  1. Open the extensions manager in Azure Data Studio. Either select the extensions icon or select Extensions in the View menu.

  2. Enter Cosmos in the search bar.

  3. Select the Azure Cosmos DB Migration for MongoDB extension and view its details.

    Screenshot of Azure Cosmos DB for MongoDB migration extension install button.

  4. Select Install.

Configure extension settings

You can configure some extension settings after installing the extension. This step is optional. If no settings are explicitly configured, the extension uses default settings.

  1. Go to extensions and select Azure Cosmos DB Migration for MongoDB, select the manage settings icon, and then select extension settings.

    Screenshot of extension settings selection.

  2. Under extension settings for this extension, provide the Assessment path to change the location where the assessment metadata is stored. If left blank, the default location is used.

Connect to the MongoDB source

Use the extension for the first time to connect to the existing MongoDB source instance. Ensure that you have the connection credentials for the source ready before starting this section.

  1. Locate the connections icon in the menu bar, and select New Connection.

    Screenshot of the connections menu bar in the extension.

  2. In the Connection pane, fill out the following fields:

    Value
    Connection type Azure Cosmos DB for MongoDB
    Connection string/Parameters Use the connection string or parameters for your existing source MongoDB instance.
    Server group Default
    Name (optional) Provide a unique name for this connection.

  3. Select Connect.

  4. Open the context menu for the new connection in the Connections pane. Then, select Manage.

    Screenshot of the Manage database screen.

  5. Select Azure Cosmos DB Migration.

    Screenshot of the Migration Assessment database screen.

Run an assessment

The assessment examines your current MongoDB data estate and utilization. The assessment feature then generates a comprehensive report that helps you identify the necessary actions to take before migrating your workloads to Azure Cosmos DB for MongoDB.

  1. Locate and navigate to the Dashboard tab. Then, select Assess and Migrate Databases(s).

    Screenshot of the Dashboard tab within the migration feature of the extension.

  2. Complete the wizard to provide details to the extension so that it can perform an assessment.

    Screenshot of assessment details before credentials are validated.

    1. In the Assessment name field, enter a title.

    2. Select the target Azure Cosmos DB for MongoDB account from the Offering dropdown.

    3. Provide the path to MongoDB Logs.

      Tip

      This is an optional field, however specifying the logs path results in more granular findings at the collection level. When the log folder isn't specified, the tool uses details from the serverStatus command to perform the assessment.

      The serverStatus command returns feature usage only since the last restart, so you would need to ensure that sufficient time has passed since the last server restart to get an assessment that accurately reflects your actual workload.

    4. Provide the path to Data assessment logs.

      Tip

      Although this field is optional, providing data assessment logs can yield more detailed insights about the workload. These logs are obtained by scanning data and reading verbose logs. The data assessment runs independently as a CLI before initiating the migration assessment, and the resulting JSON is then supplied here. Download the data assessment CLI here.

    5. Select Run validation to validate the assessment inputs.

  3. Once the validation is successful, select Start assessment to run the assessment.

    Screenshot of assessment details after credentials are validated.

  4. Depending on the size of your source server, the assessment takes a few minutes. Wait for the assessment to complete before continuing.

    Screenshot of a new assessment in progress.

  5. After the assessment is complete, you should now have an assessment report.

    Screenshot of the new assessment report for the source MongoDB instance.

  6. In the assessment report, select the instance name to review a list of server-wide issues. Select a specific database to view issues that are only applicable to the selected database.

    Screenshot of the new assessment report for the selected database within the source MongoDB instance.

  7. Select Download Report to get a consolidated downloadable report.

  8. Study the assessment report to identify any actions you need to take for a seamless migration of your workloads on Azure Cosmos DB for MongoDB. Before moving to the next step, ensure that all blocking issues reported in the assessment are handled. If there are any unresolved issues, you can exit the process and handle them later. Once the issues are resolved, you can come back and restart the assessment and migration process.

Perform a migration

Now, use the assessment report to perform a migration of your data from your source MongoDB instance to your target vCore-based Azure Cosmos DB for MongoDB account.

  1. In the assessment report screen, select any databases you plan to migrate. Then, select Next.

    Screenshot of the database selected for migration.

  2. Narrow down the lists to Select the target Azure Cosmos DB for MongoDB account by filtering by subscription and then resource group. Then provide connection credentials necessary to connect to the account.

    Screenshot of the selection of a target Azure Cosmos DB for MongoDB account.

  3. Select Test connection to validate the credentials for the Azure Cosmos DB for MongoDB account. Select Next to navigate to the mapping of collections from the source to the target.

  4. Choose either Skip or Migrate for each collection in the list of mappings. Collections that already exist in the target are automatically marked with an icon and set to Skip by default. Select Next to configure the Azure Database Migration Service (DMS).

    Screenshot of the mapping of collections from the source to the target.

    Warning

    Opting to Migrate an existing collection will overwrite the entire collection, resulting in irrecoverable data loss. Please exercise caution when choosing this option.

  5. Choose an existing Azure Database Migration Service instance from the dropdown or select Create New to create a new migration service. Azure Database Migration Service is a service that migrates data to and from Azure data platforms by using cloud infrastructure for data transfer, instead of relying on local resources.

    Screenshot of the option to choose a migration service.

  6. Select the migration mode that's most appropriate for your use case.

    • Online migration copies collection data, ensuring updates are also replicated during the process. This method is advantageous with minimal downtime, allowing continuous operations for business continuity. Use this option when ongoing operations are crucial, and reducing downtime is a priority.
    • Offline migration captures a snapshot of the database at the beginning, offering a simpler and predictable approach. It works well when using a static copy of the database is acceptable, and real-time updates aren't essential.

  7. Select Next to view the migration summary. Once you reviewed and confirmed the details, select Create Schema to create resources on the target account.

    Screenshot of option to view migration summary.

  8. Select Start Migration to initiate the data transfer using the selected migration service.

    Screenshot of the option to start a migration using the migration service.

  9. In the Confirm Settings pop-up, verify that your firewall settings meet the requirements. Select OK to confirm.

    Screenshot of the Confirm Settings pop-up, to verify your firewall settings.

The data migration tasks are run on Azure Database Migration Service hence you aren't required to be connected to the source and target environments during the data migration. The status is updated on the dashboard at frequent intervals

Note

If you selected more than 50 collections to migrate, the migration job will be batched into multiple jobs on the migration service, each containing max of 50 collections.

  1. Monitor the migration status on the dashboard page once the jobs are initialized.

    Screenshot of dashboard with migration status.

  2. To view more details, select a specific migration from the list of migrations.

Monitoring Offline migrations

The status is automatically updated once all data migration tasks are finished.

Screenshot of collections with migration status.

Once the data is successfully copied from the source to the target, offline migrations status indicates success.

Screenshot of success in migration status.

  • Change the connection string in your applications to point to the target account.

Monitoring Online migrations

Online migrations, unlike offline migrations, don't automatically complete. Instead, they run continuously until they're manually finalized by selecting Cutover.

Screenshot of online migration status.

To complete the online migration, follow these steps in the given order:

  1. The Cutover button is enabled once the Initial Data Load is completed for all collections. The migration is currently in the replication phase, continuously copying updates from the source instance to the target instance to keep it up-to-date with the latest changes.

  2. When ready to perform the migration cutover, stop all incoming transactions to the source collections being migrated.

  3. Monitor the replication changes in the table and wait until the 'Replication Changes Played' metric stabilizes. A stable 'Replication Changes Played' metric indicates that all updates from the source are successfully copied to the target.

  4. Select Cutover when the replication gap is minimal for all collections and the 'Replication Changes Played' metric is stable.

  5. Manually validate that the row count is the same between the source and target collections.

  6. In the pop-up dialog. Select Complete Cutover.
    Screenshot of online migration cutover pop-up.

    Note

    Performing the cutover operation without validating that the source and target are synced may result in data loss.

  7. Wait for the status to change to Succeeded.

  8. Update the connection string in your applications to point to the target account.

View past migrations and assessments

It's often useful to review past assessments and migrations. The extension provides an interface to review summaries of past assessment and migrations. The extension also provides the capability to review detailed historical assessment reports.

  1. To view past migrations, select the Migrations tab in the toolbar. The migrations list contains all migrations that were initiated on the current machine. You can select on a specific migration in the list to get more details.

    Screenshot of the list of all migrations created using the extension.

  2. To view past assessments, select the Assessments tab in the toolbar. The assessments list contains all assessments that were initiated on the current machine.

    Screenshot of the list of all assessments performed using the extension.

Next step

Review FAQ for the Azure Cosmos DB for MongoDB migration extension