How to retrieve Azure ml credentials within pipeline job runtime?

Question

I am using the AzureML Python SDK V2 and want to create an ML client inside a pipeline job without having to pass subscription ID or resource group as inputs. How can I achieve this?

Answer 1

@guru nath Thanks for reaching out to us, please let me know if I am not on the right direction.

In Azure ML Python SDK V2, you don't need to manually provide your Azure subscription ID or resource group to create an ML client. Instead, you can use the azure.ml.core.workspace.Workspace class to get the current workspace, and then use this workspace to initialize your client.

Here is a Python code snippet that shows how to do this:

from azure.ml.core import Workspace  
from azure.ml import MLClient  
  
# Get current workspace  
workspace = Workspace.from_config()  
  
# Create ML client with current workspace  
client = MLClient(workspace=workspace)  

In this code, Workspace.from_config() retrieves the workspace details from a file named config.json in the current directory or its parent directories. This file is automatically created when you set up your Azure ML environment and contains the subscription ID, resource group, and workspace name.

Note: For this to work, your pipeline job runtime must have access to the config.json file or you must have set the corresponding environment variables (AZUREML_WORKSPACE, AZUREML_SUBSCRIPTION, and AZUREML_RESOURCE_GROUP) in your runtime environment.

If you are running this code in Azure ML pipeline job, it will automatically have access to the current workspace without needing to provide the subscription ID or resource group. The Workspace.from_config() method will use the workspace associated with the pipeline run.

I hope this helps.

Regards,

Yutong

-Please kindly accept the answer and vote 'Yes' if you feel helpful to support the community, thanks a lot.

Answer 2

I can't seem to get this to work from within my pipeline component (credential token is needed to access the Azure ML REST API -- need to generate a data asset from my labelling job)...

Also not sure if this suggestion works with the latest python sdk? I believe the packages are now structured something like azure.ai.ml rather than azure.ml ...

I see MSI_ENDPOINT and MSI_SECRET env vars set from within my pipeline component... but have not been able to return anything useful (just a bunch of 400s)
<ClientResponse(http://localhost:46809/MSI/token/?resource=https://management.azure.com/.default&api-version=2017-09-01) [400 Bad Request]>