Can I connect directly to CosmosDB from ADF using the key vault?

Andrew Hopkinson 6 Reputation points
2024-03-06T17:59:38.0433333+00:00

I am trying to set up an azure key vault key to connect from Azure Data Factory directly to CosmosDB. I am following this link but it seems to be for a web app service.

https://learn.microsoft.com/en-us/azure/cosmos-db/store-credentials-key-vault

This creates two secret key names which doesn't work with ADF. Is there a way to connect directly to cosmos using the key vault? I am able to connect to the key vault from ADF. We tried setting up a secret value in the key vault using the "Connection String" value from cosmos but when I select this from the drop down in ADF, I get this error message:

Expecting connection string of format "key1=value1; key2=value2".

Azure Key Vault
Azure Key Vault
An Azure service that is used to manage and protect cryptographic keys and other secrets used by cloud apps and services.
1,124 questions
Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,448 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,599 questions
{count} votes

3 answers

Sort by: Most helpful
  1. Dillon Silzer 54,646 Reputation points
    2024-03-07T01:12:09.0133333+00:00

    Hi Andrew,

    Try following the guide below to get key Vault secrets:

    Use Azure Key Vault secrets in pipeline activities

    https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-azure-key-vault-secrets-pipeline-activities

    The tutorial you gave is in fact a guide for Azure Web App to store and use credentials stored in Azure KV.

    If this is helpful please accept answer.


  2. Andrew Hopkinson 6 Reputation points
    2024-03-07T17:56:06.4166667+00:00

    I figure it out myself, the issue was that the connection string copied from the the cosmos resource did not include the database name. I had to include ";Databasename=[cosmosdbname]" to end of the secret value in the Azure Key Vault and it worked. This is not well documented and the error message is not clear. The only way I found this was by hovering over the very tiny "i" beside the secret word in ADF for the key vault authentication.


  3. JamesTran-MSFT 36,376 Reputation points Microsoft Employee
    2024-03-07T18:29:54.8866667+00:00

    @Andrew Hopkinson

    I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this!

    Issue:

    Can I connect directly to CosmosDB from ADF using the key vault?

    I am trying to set up an azure key vault key to connect from Azure Data Factory directly to CosmosDB. I am following this link but it seems to be for a web app service.

    This creates two secret key names which doesn't work with ADF. Is there a way to connect directly to cosmos using the key vault? I am able to connect to the key vault from ADF. We tried setting up a secret value in the key vault using the "Connection String" value from cosmos but when I select this from the drop down in ADF,

    I get this error message: Expecting connection string of format "key1=value1; key2=value2".

    Solution:

    The issue was that the connection string copied from the the cosmos resource did not include the database name. I had to include ";Databasename=[cosmosdbname]" to end of the secret value in the Azure Key Vault and it worked.

    This is not well documented and the error message is not clear. The only way I found this was by hovering over the very tiny "i" beside the secret word in ADF for the key vault authentication.

    If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

    I hope this helps!

    If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

    0 comments No comments