az ml job

Note

This reference is part of the ml extension for the Azure CLI (version 2.15.0 or higher). The extension will automatically install the first time you run an az ml job command. Learn more about extensions.

Manage Azure ML jobs.

An Azure ML job executes a task against a specified compute target. You can configure jobs to scale out model training on Azure. Azure ML supports different job types with different capabilities. For example, the most basic job, a command job, executes a command in a Docker container and can be leveraged for single-node and distributed training. A sweep job executes a hyperparameter sweep over a specified search space for tuning a model's hyperparameters.

Jobs also enable systematic tracking for your ML experimentation and workflows. Once a job is created, Azure ML maintains a run record for the job that includes the metadata, any metrics, logs, and artifacts generated during the job, code that was executed, and the Azure ML environment used. All of your jobs' run records can be viewed in Azure ML studio.

Commands

az ml job archive

Archive a job.

az ml job cancel

Cancel a job.

az ml job create

Create a job.

az ml job download

Download all job-related files.

az ml job list

List jobs in a workspace.

az ml job restore

Restore an archived job.

az ml job show

Show details for a job.

az ml job stream

Stream job logs to the console.

az ml job update

Update a job.

az ml job archive

Archive a job.

Archiving a job will hide it by default from list queries (az ml job list). You can still continue to reference and use an archived job in your workflows. Only completed jobs can be archived.

az ml job archive --name
                  --resource-group
                  --workspace-name

Required Parameters

--name -n

Name of the job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

az ml job cancel

Cancel a job.

az ml job cancel --name
                 --resource-group
                 --workspace-name

Required Parameters

--name -n

Name of the job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

az ml job create

Create a job.

To create a job, you will typically need to configure any code to be run, an environment encapsulating the dependencies, a compute target to execute the job on, and any additional job-specific settings. When a job is created, it is submitted for execution against the specified compute resource.

az ml job create --file
                 --resource-group
                 --workspace-name
                 [--name]
                 [--save-as]
                 [--set]
                 [--skip-validation]
                 [--stream]
                 [--web]

Examples

Create a job from a YAML specification file

az ml job create --file job.yml --resource-group my-resource-group --workspace-name my-workspace

Create a job from a YAML specification file and open the job's run details in the Azure ML studio portal

az ml job create --file job.yml --web --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--file -f

Local path to the YAML file containing the Azure ML job specification. The YAML reference docs for job can be found at: https://aka.ms/ml-cli-v2-job-command-yaml-reference, https://aka.ms/ml-cli-v2-job-sweep-yaml-reference, https://aka.ms/ml-cli-v2-job-pipeline-yaml-reference.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--name -n

Name of the job.

--save-as -a

File to which the created job's state in YAML format will be written.

--set

Update an object by specifying a property path and value to set. Example: --set property1.property2=.

--skip-validation

Skip validation in creating the resource. Note that dependent resources will not skip their validation in creating.

--stream -s

Indicates whether to stream the job's logs to the console.

--web -e

Show the job's run details in Azure ML studio in a web browser.

az ml job download

Download all job-related files.

The files will be downloaded in a folder named after the job's name.

az ml job download --name
                   --resource-group
                   --workspace-name
                   [--all]
                   [--download-path]
                   [--output-name]

Examples

Download a job's logs and outputs to the current working directory

az ml job download --name my-job --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--name -n

Name of the job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--all

Download all the outputs of the job.

--download-path -p

Path to download the job files to. If omitted, job files will be downloaded to the current directory.

--output-name

The name of the user-defined output to download. This should correspond to a key in the outputs dictionary of a job. If omitted, the job's default artifact output files will be downloaded.

az ml job list

List jobs in a workspace.

az ml job list --resource-group
               --workspace-name
               [--all-results {false, true}]
               [--archived-only]
               [--include-archived]
               [--max-results]
               [--parent-job-name]

Examples

List all the jobs status in a workspace using --query argument to execute a JMESPath query on the results of commands.

az ml job list --query "[].{Name:name,Jobstatus:status}"  --output table --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--all-results

Returns all results.

accepted values: false, true
--archived-only

List archived jobs only.

--include-archived

List archived jobs and active jobs.

--max-results -r

Max number of results to return. Default is 50.

default value: 50
--parent-job-name -p

Name of the parent job. Will list all jobs whose parent_job_name matches the given name.

az ml job restore

Restore an archived job.

When an archived job is restored, it will no longer be hidden from list queries (az ml job list).

az ml job restore --name
                  --resource-group
                  --workspace-name

Required Parameters

--name -n

Name of the job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

az ml job show

Show details for a job.

az ml job show --name
               --resource-group
               --workspace-name
               [--web]

Examples

Show the status of a job using --query argument to execute a JMESPath query on the results of commands.

az ml job show --name my-job-id --query "{Name:name,Jobstatus:status}"  --output table --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--name -n

Name of the job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--web -e

Show the job's run details in Azure ML studio in a web browser.

az ml job stream

Stream job logs to the console.

az ml job stream --name
                 --resource-group
                 --workspace-name

Required Parameters

--name -n

Name of the job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

az ml job update

Update a job.

Only the 'tags' and 'properties' properties can be updated.

az ml job update --name
                 --resource-group
                 --workspace-name
                 [--add]
                 [--force-string]
                 [--remove]
                 [--set]
                 [--web]

Required Parameters

--name -n

Name of the job.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--add

Add an object to a list of objects by specifying a path and key value pairs. Example: --add property.listProperty <key=value, string or JSON string>.

--force-string

When using 'set' or 'add', preserve string literals instead of attempting to convert to JSON.

--remove

Remove a property or an element from a list. Example: --remove property.list OR --remove propertyToRemove.

--set

Update an object by specifying a property path and value to set. Example: --set property1.property2=.

--web -e

Show the job's run details in Azure ML studio in a web browser.