Connection switching is not happening internally between the service principal certificate key and Managed identity in Synapse notebook for the ADLS account

Question

Connection switching is not happening internally between the service principal certificate key and Managed identity in Synapse notebook for the ADLS account

Venkateswarlu Pottapalli 0 Microsoft External Staff

Hi,

I am connecting to the source using the linked service with service principal certificate key in azure synapse notebook. I am able to pull the data in data frame but not able to load the data frame in ADLS Gen2 which is mapped to the same Azure synapse Analytics where I am running the notebook.

The error is like permission issue to the managed identity for the ADLS account.

I am able to load the data using the same notebook if I don't use the connection without service principal certificate. you can refer the two scenarios below.

Scenario 1: (data write is working fine)

step1 : Connect to source ADLS using linked service (managed identity) ----Successful

Step 2 : Create data frame using step1 ----Successful

Step3 : load data in ADLS Gen2 which mapped to Azure synapse Analytics where I am running notebook. --Successful

Scenario 2: (Data write is not working)

step1 : Connect to source ADLS using linked service (Service Principal with certificatekey) --Successful

Step 2 : Create data frame using step1 --Successful

Step3 : load data in ADLS Gen2 which is mapped to Azure synapse Analytics where I am running notebook. --Failed

Based on my understanding it's not able to switch connection from Certificate connected ADLS to managed identity connected ADLS connection.

Please help me what could be the solution for it.

Regards,

Venkat

Ganesh Gurram 6,050 Reputation points Microsoft External Staff

2025-03-20T04:37:49.3+00:00

@Venkateswarlu Pottapalli

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Venkateswarlu Pottapalli 0 Reputation points Microsoft External Staff

2025-03-20T04:45:22.6233333+00:00

Hi thanks for the response,

but getting the error message like **"**Unable to load OAuth token provider class.Configuration property fs.azure.account.oauth2.msi.tenant not found."
Ganesh Gurram 6,050 Reputation points Microsoft External Staff

2025-03-20T07:17:11.9966667+00:00
@Venkateswarlu Pottapalli

The error "Unable to load OAuth token provider class. Configuration property fs.azure.account.oauth2.msi.tenant not found." indicates that some required authentication configurations are missing.

Add the following additional configuration in your notebook before writing data:

storage_account_name = "<your_storage_account_name>" tenant_id = "<your_tenant_id>" spark.conf.set(f"fs.azure.account.auth.type.{storage_account_name}.dfs.core.windows.net", "OAuth") spark.conf.set(f"fs.azure.account.oauth.provider.type.{storage_account_name}.dfs.core.windows.net", "org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider") spark.conf.set(f"fs.azure.account.oauth2.msi.tenant.{storage_account_name}.dfs.core.windows.net", tenant_id)

Replace <your_storage_account_name> with your actual ADLS Gen2 storage account name and <your_tenant_id> with your Azure Active Directory (AAD) Tenant ID.

I hope this helps.
phemanth 15,235 Reputation points Microsoft External Staff

2025-03-21T06:49:50.25+00:00

@Venkateswarlu Pottapalli was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Venkateswarlu Pottapalli 0 Reputation points Microsoft External Staff

2025-04-16T21:52:38.44+00:00

@phemanth , getting the same issue. I didn't get any solution for it

1 answer

Your answer

Ganesh Gurram 6,050 Reputation points Microsoft External Staff

2025-03-20T04:37:49.3+00:00

@Venkateswarlu Pottapalli

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Venkateswarlu Pottapalli 0 Reputation points Microsoft External Staff

2025-03-20T04:45:22.6233333+00:00

Hi thanks for the response,

but getting the error message like **"**Unable to load OAuth token provider class.Configuration property fs.azure.account.oauth2.msi.tenant not found."
Ganesh Gurram 6,050 Reputation points Microsoft External Staff

2025-03-20T07:17:11.9966667+00:00

@Venkateswarlu Pottapalli

The error "Unable to load OAuth token provider class. Configuration property fs.azure.account.oauth2.msi.tenant not found." indicates that some required authentication configurations are missing.

Add the following additional configuration in your notebook before writing data:

storage_account_name = "<your_storage_account_name>" tenant_id = "<your_tenant_id>" spark.conf.set(f"fs.azure.account.auth.type.{storage_account_name}.dfs.core.windows.net", "OAuth") spark.conf.set(f"fs.azure.account.oauth.provider.type.{storage_account_name}.dfs.core.windows.net", "org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider") spark.conf.set(f"fs.azure.account.oauth2.msi.tenant.{storage_account_name}.dfs.core.windows.net", tenant_id)

Replace <your_storage_account_name> with your actual ADLS Gen2 storage account name and <your_tenant_id> with your Azure Active Directory (AAD) Tenant ID.

I hope this helps.
phemanth 15,235 Reputation points Microsoft External Staff

2025-03-21T06:49:50.25+00:00

@Venkateswarlu Pottapalli was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Venkateswarlu Pottapalli 0 Reputation points Microsoft External Staff

2025-04-16T21:52:38.44+00:00

@phemanth , getting the same issue. I didn't get any solution for it

Answer 1

@Venkateswarlu Pottapalli

It looks like the notebook session continues to use the Service Principal (certificate key) authentication and is not switching to the Managed Identity when writing data to ADLS Gen2.

To resolve this, please follow these steps:

Explicitly Configure Spark to Use Managed Identity - Before writing data to ADLS Gen2, ensure your Spark session is set to use the Managed Identity by adding the following configuration:

storage_account_name = "<your_storage_account_name>"

spark.conf.set(f"fs.azure.account.auth.type.{storage_account_name}.dfs.core.windows.net", "OAuth")
spark.conf.set(f"fs.azure.account.oauth.provider.type.{storage_account_name}.dfs.core.windows.net", "org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider")

Replace <your_storage_account_name> with your actual ADLS Gen2 storage account name.

Verify Managed Identity Permissions on ADLS Gen2 - Ensure that your Synapse workspace’s Managed Identity has the Storage Blob Data Contributor role assigned at the storage account level. You can do this by:

Navigating to your ADLS Gen2 Storage Account in the Azure Portal.
Selecting Access Control (IAM) > Add Role Assignment.
Assigning the Storage Blob Data Contributor role to your Synapse Managed Identity.

For more details refer: Grant permissions to managed identity in Synapse workspace

Troubleshoot connector and format issues in mapping data flows in Azure Data Factory

I hope this information helps.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Share via

Connection switching is not happening internally between the service principal certificate key and Managed identity in Synapse notebook for the ADLS account

1 answer

Your answer