How to connect Azure ML to an ADLS gen 2 container as a Datastore?

Question

How to connect Azure ML to an ADLS gen 2 container as a Datastore?

Neal Lockhart 40

Hi -

I'm working on trying to connect my ADLS gen 2 to Azure ML. It seems the only way to create an ADLS gen 2 datastore is via service principal, is this accurate? It is not possible to create an ADLS gen 2 datastore via SAS or account key?

If it is not possible to create a datastore via other authentication methods, do we have to create a new "app registration" and link that to Azure ML + ADLS Gen 2?

These were the instructions I received but it seems very roundabout that there isn't a way to just create a datastore from ADLS gen2 using other authentication methods, so I assume I must be overlooking something.

Might be worth noting this is all being done via azure portal.

Ravada Shivaprasad 375 Reputation points Microsoft External Staff Moderator

2025-05-19T20:03:00.1333333+00:00
Hi Neal Lockhart

When connecting Azure Data Lake Storage (ADLS) Gen 2 to Azure Machine Learning, service principal authentication is the recommended and required method for datastore creation. Unlike other Azure storage services, ADLS Gen 2 does not support SAS tokens or account keys for creating a datastore in Azure Machine Learning. This restriction is due to enterprise-grade security requirements, ensuring persistent and controlled access to storage resources.

Microsoft's official documentation confirms that ADLS Gen 2 datastores must be configured using service principal authentication, as outlined here:

Create an Azure Machine Learning datastore

Manage credentials for secure authentication

By using a service principal, you establish a secure, scalable integration between Azure Machine Learning and ADLS Gen 2 while maintaining strict security controls over data access and permissions.
Since service principal authentication is required for creating an Azure Data Lake Storage (ADLS) Gen 2 datastore in Azure Machine Learning, registering an application in Azure Active Directory (AAD) is necessary. This app registration enables secure credential management and facilitates controlled access between Azure ML and ADLS Gen 2. The process involves creating an app registration within Azure AD, generating a client secret under Certificates & secrets, and assigning the necessary API permissions, including at least the Storage Blob Data Reader role. Once the service principal is established, it must be linked to the datastore in Azure Machine Learning by providing the Tenant ID, Client ID, and Client Secret during the connection setup. This authentication method ensures enterprise-grade security, enabling persistent access while preventing unauthorized credentials usage.

For detailed guidance, refer to the official Microsoft documentation:

How to register an app in Azure AD

Configuring a datastore in Azure Machine Learning

Managing role-based access control for ADLS Gen 2

This structured approach ensures secure authentication while maintaining scalable data access between Azure Machine Learning and ADLS Gen 2 for machine learning workflows.
Yes, your concern is addressed in the earlier response. Azure Data Lake Storage (ADLS) Gen 2 datastores in Azure Machine Learning require service principal authentication, and other methods—such as SAS tokens or account keys—are not supported for datastore creation. This restriction is due to security and compliance requirements, ensuring persistent, controlled access for enterprise environments.

While it may seem like a roundabout approach, the authentication process is designed to maintain strict access control and mitigate security risks. Registering an app in Azure Active Directory (AAD) and linking it via a service principal is the necessary step to establish a secure connection. If you need further clarification or troubleshooting assistance, let me know!

Thanks
Neal Lockhart 40 Reputation points

2025-05-19T20:09:46.26+00:00

Thank you! That all makes sense. For my own knowing and for future folks, could you please link me to the documentation where this process is detailed? I've looked in the links you've sent and I don't see where setting up the SP for ADLS gen 2 connection to Azure ML is detailed specifically.

I see where it mentions that you can use SP, but I don't see where it shows you need to add a new app to Azure AD to make that work.

Thank you!
Neal Lockhart 40 Reputation points

2025-05-20T20:24:52.31+00:00

@Ravada Shivaprasad perfect, thank you so much! I now see the exact process that needs to be followed.
SriLakshmi C 5,930 Reputation points Microsoft External Staff Moderator

2025-05-21T12:33:32.69+00:00

Hi Neal Lockhart

I'm glad to hear that your issue has been resolved! I’ve converted Ravada Shivaprasad earlier comment into an actual answer, If this answers your query, please do click "Accept the answer” for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thank you!

1 answer

Your answer

Neal Lockhart 40 Reputation points

2025-05-19T20:09:46.26+00:00

Thank you! That all makes sense. For my own knowing and for future folks, could you please link me to the documentation where this process is detailed? I've looked in the links you've sent and I don't see where setting up the SP for ADLS gen 2 connection to Azure ML is detailed specifically.

I see where it mentions that you can use SP, but I don't see where it shows you need to add a new app to Azure AD to make that work.

Thank you!
Neal Lockhart 40 Reputation points

2025-05-20T20:24:52.31+00:00

@Ravada Shivaprasad perfect, thank you so much! I now see the exact process that needs to be followed.
SriLakshmi C 5,930 Reputation points Microsoft External Staff Moderator

2025-05-21T12:33:32.69+00:00

Hi Neal Lockhart

I'm glad to hear that your issue has been resolved! I’ve converted Ravada Shivaprasad earlier comment into an actual answer, If this answers your query, please do click "Accept the answer” for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

Thank you!

Answer 1

Hi Neal Lockhart

You're absolutely right Neal, To seek more precise documentation on setting up a service principal (SP) for ADLS Gen 2 in Azure Machine Learning. While the previous links mention SP authentication, they don't explicitly detail the app registration process in Azure AD for this specific integration.

Here are some relevant Microsoft resources that provide guidance:

Access ADLS Gen2 data from Azure Machine Learning – This tutorial outlines how to create an Azure Machine Learning Datastore that connects to ADLS Gen2.
Azure Data Lake Storage Gen2 Service Account Connectivity – This guide explains how to configure service principal authentication with Azure AD (formerly Entra ID), including app registration and role assignments.

These sources should help clarify the end-to-end setup, including registering an app in Azure AD, assigning permissions, and linking it to Azure ML for secure authentication. Let me know if you need further details!

Disclaimer: This response contains a reference to a third-party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

Thanks

Share via

How to connect Azure ML to an ADLS gen 2 container as a Datastore?

1 answer

Your answer