How to create Pipeline endpoint in Azure machine learning using CLI (v2) SDK

Question

In Azure ML, using python SDK we are able to create and publish pipeline endpoints successfully.

We are trying to create a pipeline endpoint using CLI (v2) but, we are only able to create job using YAML specification file.

az ml job create

Is there any document or command/code available for creating and publish pipeline endpoints in Azure ML using CLI (v2).

Answer 1

Hello @SHAIKH, Alif Abdul

Thanks for reaching out to us. There are some changes for the CLI V2 compared to V1.

The Azure Machine Learning CLI v2 (CLI v2) is the latest extension for the Azure CLI. The CLI v2 provides commands in the format az ml <noun> <verb> <options> to create and maintain Azure Machine Learning assets and workflows. The assets or workflows themselves are defined using a YAML file. The YAML file defines the configuration of the asset or workflow – what is it, where should it run, and so on.

A few examples of CLI v2 commands:

• az ml job create --file my_job_definition.yaml

Please see this document for more information - https://learn.microsoft.com/en-us/azure/machine-learning/concept-v2?view=azureml-api-2#azure-machine-learning-cli-v2

For how to submit the training job, please refer to this guidance -

https://learn.microsoft.com/en-us/azure/machine-learning/how-to-train-model?view=azureml-api-2&tabs=azurecli#4-submit-the-training-job

The az ml job create command used in this example requires a YAML job definition file. The contents of the file used in this example are:

Note

To use serverless compute (preview), delete compute: azureml:cpu-cluster" in this code.

$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json
code: src
command: >-
  python main.py 
  --iris-csv ${{inputs.iris_csv}}
  --C ${{inputs.C}}
  --kernel ${{inputs.kernel}}
  --coef0 ${{inputs.coef0}}
inputs:
  iris_csv: 
    type: uri_file
    path: wasbs://******@azuremlexamples.blob.core.windows.net/iris.csv
  C: 0.8
  kernel: "rbf"
  coef0: 0.1
environment: azureml:AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest
compute: azureml:cpu-cluster
display_name: sklearn-iris-example
experiment_name: sklearn-iris-example
description: Train a scikit-learn SVM on the Iris dataset.

In the above, you configured:

code - path where the code to run the command is located

command - command that needs to be run

inputs - dictionary of inputs using name value pairs to the command. The key is a name for the input within the context of the job and the value is the input value. Inputs are referenced in the command using the ${{inputs.<input_name>}} expression.

environment - the environment needed to run the training script. In this example, we use a curated or ready-made environment provided by Azure Machine Learning called AzureML-sklearn-0.24-ubuntu18.04-py37-cpu. We use the latest version of this environment by using the @latest directive. You can also use custom environments by specifying a base docker image and specifying a conda yaml on top of it. To submit the job, use the following command. The run ID (name) of the training job is stored in the $run_id variable:

run_id=$(az ml job create -f jobs/single-step/scikit-learn/iris/job.yml --query name -o tsv)

You can use the stored run ID to return information about the job. The --web parameter opens the Azure Machine Learning studio web UI where you can drill into details on the job:

az ml job show -n $run_id --web

I hope this helps, please refer to the document and let me know if you have any questions.

Regards,

Yutong

-Please kindly accept the answer and vote 'Yes' if you feel helpful to support the community, thanks a lot.