Troubleshooting Data Factory Connectivity to Cosmos DB for Data Migration

Question

I'm working on transferring data from a Cosmos DB account to Azure Storage using a pipeline for migration. However, I'm facing an issue where Data Factory is unable to connect to the Cosmos DB account. We've restricted access to specific IP address ranges, and though I tried adding the IP addresses of the Data Factory, the pipeline seems to use different IP addresses each time. How can I ensure that Data Factory consistently accesses the Cosmos DB account?

Answer 1

Hi @Glasier

Welcome to Microsoft Q&A platform and thanks for posting your question here.

Based on the issue you're facing, it seems like you've restricted access to specific networks and haven't selected 'Accept connections from within public Azure data centers.

When you choose 'Accept connections from within public Azure data centers,' all Azure services should be able to connect from within those data centers.

Alternatively, it could be that your Data Factory is not in the same Virtual Network (Vnet) as the Cosmos DB account.

To address this:

Method 1:

• Add the Data Factory to the same Vnet or whitelist its IP address. Note that ADF may keep adding IP addresses, making it challenging to manage.
• The recommended approach is to configure the Vnet service endpoint for both Data Factory and Cosmos DB:
  • Create a Vnet and Subnet.
  • Add a firewall rule in Cosmos DB to allow traffic only from the Vnet's subnet.
  • Configure the ADF to use the same Vnet and subnet.

Method 2:

• Use Azure Private Link to securely access your Cosmos DB account from your Data Factory. This allows you to access Azure PaaS Services, like Cosmos DB, over a private endpoint in your virtual network.

Refer to the documentation for private endpoints: Managed Virtual Network Private Endpoint

For configuring the Cosmos DB firewall, follow this guide: How to configure firewall for Cosmos DB

I hope this information is helpful. Feel free to ask if you have any further questions.