Synapse Scala | Read Cosmos DB Container data using managed identity

AJITH KUMAR RAI 0 Reputation points
2024-05-01T14:01:16.22+00:00

Hi Team,

I tried reading Cosmos data using Managed Identity inside a Synapse Scala notebook, but it's not working as expected.

Already provided access for synapse workspace by using below command

New-AzCosmosDBSqlRoleAssignment -ResourceGroupName <Resourcegroupname> -AccountName <CosmosAccountName> -RoleDefinitionName "Cosmos DB Built-in Data Contributor" -PrincipalId <WorkspaceMangedidentity> -Scope "/"

Please find below steps and error message we are getting.

I created linked service as shown below screen.

User's image

as per this article: https://learn.microsoft.com/en-us/azure/synapse-analytics/synapse-link/how-to-query-analytical-store-spark-3

I created below code block inside my notebook

val linkedServiceName = "MSICosmosDbNoSql1"
val secretlinkedsvc="SecretCosmosDbNoSql1"
val containerName = "Items"

    val df = spark.read.format("cosmos.olap")
      .option("spark.synapse.linkedService", linkedServiceName)
      .option("spark.cosmos.container", containerName)
      .load()

But it is not working with linked service: MSICosmosDbNoSql1

And it is working with SecretCosmosDbNoSql1

Here we want to connect by using Managed identity only.

Just for info: We also Enable Azure Synapse Link at Cosmos side.

Error Message:

Caused by: com.microsoft.azure.synapse.TokenServiceClientResponseStatusException: Token Service returned 'Client Error' (400), with message: {"result":"DependencyError","errorId":"BadRequest","errorMessage":"[Code=AuthTypeNotSupported, Target=MSICosmosDbNoSql1, Message=Linked Service Auth Type not supported]. TraceId : da03309d-ee99-4b1e-8cfd-f39f73dd3b6f | client-request-id : a363ee69-ab48-487d-abda-6e2e9de40cc8. Error Component : LSR"}

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,457 questions
Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,469 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Smaran Thoomu 10,635 Reputation points Microsoft Vendor
    2024-05-06T07:46:28.3866667+00:00

    @AJITH KUMAR RAI Thank you for providing additional information. Based on your response, it seems that you have already tried several approaches to resolve the issue, but you are still encountering the "AuthTypeNotSupported" error when trying to read data with the notebook using the MSI linked service.

    It's possible that the "AuthTypeNotSupported" error is caused by a limitation in the current implementation of Synapse Analytics. According to the ticket you mentioned, this functionality may not be supported at this time.

    However, there are a few workarounds you can try:

    1. Use a different authentication method: Instead of using MSI, you can try using a different authentication method, such as a shared access signature (SAS) token or a connection string. You can create a new linked service with the appropriate authentication method and test the Scala code block with the new linked service.
    2. Use a different notebook: You can try using a different notebook to see if the issue is specific to the current notebook. You can create a new notebook and test the Scala code block with the new notebook.
    3. Reach out to Azure support: If the issue persists, you can try reaching out to Azure support for further assistance. They may be able to provide additional insights or workarounds to help you resolve the issue.

    I hope this helps. Do let us know if you any further queries.

    0 comments No comments