Azure Data Factory Copy Data issue while copy data into Azure synapse dedicated SQL pool

Lokesh Rajpoot 36 Reputation points
2022-10-19T07:42:32.92+00:00

I am facing error while copy data from SAP table to Azure synapse dedicated sql pool using polybase and auto create table option:-

251941-error-copy.png
Error details
Error code
2200
Troubleshooting guide
Failure type
User configuration issue
Details
ErrorCode=PolybaseOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error happened when loading data into SQL Data Warehouse. Operation: 'Create external table'.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Data.SqlClient.SqlException,Message=External file access failed due to internal error: 'Error occurred while accessing HDFS: Java exception raised on call to HdfsBridge_IsDirExist. Java exception message:
HdfsBridge::isDirExist - Unexpected error encountered checking whether directory exists or not: AbfsRestOperationException: Operation failed: "This request is not authorized to perform this operation.", 403, HEAD, https://azsfbcnindevadls.dfs.core.windows.net/datalake/?upn=false&action=getAccessControl&timeout=90',Source=.Net SqlClient Data Provider,SqlErrorNumber=105019,Class=16,ErrorCode=-2146232060,State=1,Errors=[{Class=16,Number=105019,State=1,Message=External file access failed due to internal error: 'Error occurred while accessing HDFS: Java exception raised on call to HdfsBridge_IsDirExist. Java exception message:
HdfsBridge::isDirExist - Unexpected error encountered checking whether directory exists or not: AbfsRestOperationException: Operation failed: "This request is not authorized to perform this operation.", 403, HEAD, https://azsfbcnindevadls.dfs.core.windows.net/datalake/?upn=false&action=getAccessControl&timeout=90',},],'
Source
Pipeline
PL_MARA_2017
Monitor
Copy activity
Copy data1

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,925 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. AnnuKumari-MSFT 31,731 Reputation points Microsoft Employee
    2022-10-20T09:06:12.767+00:00

    Hi @Lokesh Rajpoot ,
    Thankyou for using Microsoft Q&A platform and thanks for posting your question here.

    Could you please make sure you have Storage Blob Data Contributor role and Storage Data Reader role assigned to the Synapse managed identity in the storage account.

    In case the error still persists even after assigning the above roles, then kindly go ahead with the following steps :

    Create a database master key if one does not already exist, using your own password. This key is used to encrypt the credential secret in next step.

    CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'S0me!nfo';  
    

    Create a database scoped credential with Azure storage account key as the secret.

    CREATE DATABASE SCOPED CREDENTIAL AzureStorageCredential  
    WITH  
    IDENTITY = '<my_account>'  
    , SECRET = '<azure_storage_account_key>';  
    

    Create an external data source with CREDENTIAL option.

    CREATE EXTERNAL DATA SOURCE MyAzureStorage  
    WITH  
    ( LOCATION = 'wasbs://daily@logs.blob.core.windows.net/'  
    , CREDENTIAL = AzureStorageCredential  
    , TYPE = HADOOP  
    );  
    

    For more details, kindly check : When Attempting to Access External Table via Polybase

    Hope it helps. Please let us know how it goes.