Error in Azure Synapse Notebook Unable to Authenticate to Access Azure Managed Identity
I am trying to authenticate to access other Azure resource (Azure Digital Twins) from Azure Synapse without explicitly using secrets, so I tried to use the Azure Managed Identity and followed this tutorial. After following the tutorial, I am getting the error below (unable to authenticate):
ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable. No identity has been assigned to this resource.
SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
VisualStudioCodeCredential: Failed to get Azure user details from Visual Studio Code.
AzureCliCredential: Azure CLI not found on path
Below is the code snippet:
from azure.identity import DefaultAzureCredential
from azure.digitaltwins.core import DigitalTwinsClient
# azure_managed_identity_client_id, adt_url change to corresponding values
credential = DefaultAzureCredential(managed_identity_client_id=<azure_managed_identity_client_id>)
service_client = DigitalTwinsClient(<adt_url>, credential)
relationship_query = 'SELECT * FROM RELATIONSHIPS'
relationships = service_client.query_twins(relationship_query)
relationships_df = pd.DataFrame()
for relationship in relationships:
print(relationship)
Steps to reproduce:
- Create a managed identity instance in Azure Portal
- In access control (IAM) of the other Azure resource (ADT in our case; blob storage in demo case), grant access to the managed identity created in step 1
- Go to the identity pane of the destination service (Synapse in our case; Azure Functions in demo case) in Azure Portal to add the user assigned identity (in this case, we add the managed identity instance)
- Run the code snippet above in Azure Synapse notebook attached to a Spark Pool
Is there anything I did incorrectly? What's the best practice for accessing Azure resources that don't support Linked Service in Synapse without referencing the secret in the code (inside the Synapse notebook)?