Synapse Notebook does not run sample code

David Leach 1 Reputation point
2022-08-16T12:42:43.723+00:00

When running the sample code "Explore NYC Yellow Taxi Data using Spark" in a PySpark notebook the Spark Pool starts successfully but the code just hangs and eventually returns with the following all though it has also returned with different errors.

I'm guessing this has something to do with the fact out storage account is locked down to specific Virtual Networks and Specific IP addresses. I have followed the steps to create a Managed Private Endpoint which points to the storage account and I can see the storage account in Data Browser on the left in Synapse Studio. I have also used the PowerShell command Add-AzStorageAccountNetworkRule to enable AAD pass through as per link: https://azureaggregator.wordpress.com/2021/01/08/storage-configuration-for-external-table-is-not-accessible-while-query-on-serverless-2/ but nothing has worked. Can anyone help on this pleaase?

Error Returned

"Resolving access token for scope "https://storage.azure.com/.default" using identity of type "USER".
Credentials are not provided to access data from the source. Please sign in using identity with required permission granted.
Interactive sign-in timeout: 120 sec.
To change the sign-in tenant, restart the session with tenant ID set to environment variable "AZUREML_DATA_ACCESS_TENANT_ID" before sign in.
To always use device code for interactive sign-in, set environment variable "AZUREML_DATA_ACCESS_USE_DEVICE_CODE" to "true".
To configure timeout, set environment variable "AZUREML_DATA_ACCESS_INTERACT_TIMEOUT" to the number of seconds."

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
2,853 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. BhargavaGunnam-MSFT 10,331 Reputation points Microsoft Employee
    2022-08-17T22:34:12.133+00:00

    Hello @David Leach ,
    Thanks for the question and using MS Q&A platform.

    Please check if your Synapse workspace service principal has RBAC- "Storage blob data contributor" access on the storage account.

    You can provide access from the storage account IAM page.

    232120-image.png

    also, can you Grant your Azure Synapse workspace access to your secure storage account as a trusted Azure service?

    Here is the document: https://learn.microsoft.com/en-us/azure/synapse-analytics/security/connect-to-a-secure-storage-account#grant-your-azure-synapse-workspace-access-to-your-secure-storage-account-as-a-trusted-azure-service

    I hope this helps.