Which identity is used by test connection when creating linked service in Azue Data Factory ?

Question

My Data Factory is configured to connect a Azure Data Lake Gen2, e.g., adlss://container01@targetstorage.dfs.core.windows.net/folder01. The system assigned managed identity of my Data Factory has read permission to this Data Lake folder.

I created a linked service of this Data Lake with following configurations:

• Authenticaton type: System assigned managed identity
• URL: https://targetstorage.dfs.core.windows.net
• Test connection: To file path, container01/folder01

And I got this error: ADLS Gen2 operation failed for: Operation returned an invalid status code 'Unauthorized'.

I am wondering which identity is used during the test connection, the managed identity of the Data Factory or my AAD account configuring the Data Factory?

Answer 1

When you test the connection to the Data Lake Gen2 from the linked service in Azure Data Factory, it uses the managed identity of the Data Factory by default.

Since you have configured the linked service to use the system-assigned managed identity of the Data Factory, the test connection will use this identity to connect to the Data Lake.

The "Unauthorized" error message indicates that the managed identity does not have the necessary permissions to access the specified folder in the Data Lake.

To resolve this issue, you need to grant the necessary permissions to the managed identity of the Data Factory.

You have already mentioned that you have granted read permissions to the managed identity on the folder in the Data Lake. However, you also need to make sure that the managed identity has the necessary permissions on the container level as well.

You can try granting the Storage Blob Data Reader role to the managed identity at the container level in the Access Control (IAM) settings of the Data Lake storage account. This should allow the managed identity to read data from the specified folder in the Data Lake.

Once you have granted the necessary permissions to the managed identity, you can try testing the connection again from the linked service in Azure Data Factory

Answer 2

When you create a linked service in Azure Data Factory, you are essentially creating a connection to an external data source or service, such as a database, a file system, or a cloud storage service. When you configure the linked service, you need to provide authentication details, such as a username and password, a client ID and secret, or an access key, depending on the type of service you are connecting to.

To test the connection to the linked service, Azure Data Factory provides a Test Connection feature. When you click on the Test Connection button, Azure Data Factory will attempt to connect to the linked service using the authentication details you provided.

For some types of linked services, such as Azure Blob Storage and Azure SQL Database, Azure Data Factory can use the Managed Identity of the Azure Data Factory instance to authenticate the connection. A Managed Identity is a service principal that is automatically created and managed by Azure, and can be used to authenticate with other Azure services without the need for a username and password. When you use a Managed Identity to authenticate the connection, the Test Connection feature will use the identity of the Azure Data Factory instance to connect to the linked service.

However, for other types of linked services, such as Amazon S3 or Salesforce, the Test Connection feature may require you to provide additional authentication details, such as a client ID and secret or username and password. In these cases, the Test Connection feature will use the authentication details you provided to connect to the linked service.

In summary, the identity used by the Test Connection feature in Azure Data Factory depends on the type of linked service you are creating and the authentication method you choose to use. Using Managed Identities can simplify authentication for some types of linked services, but additional authentication details may be required for other services.