az ml online-endpoint

Note

This reference is part of the ml extension for the Azure CLI (version 2.15.0 or higher). The extension will automatically install the first time you run an az ml online-endpoint command. Learn more about extensions.

Manage Azure ML online endpoints.

Azure ML endpoints provide a simple interface for creating and managing model deployments. Each endpoint can have one or more deployments, enabling the traffic from a single scoring endpoint to be served to multiple deployments if needed. This is useful for scenarios such as controlled rollout.

Azure ML supports two types of endpoints: online and batch. Online endpoints support real-time inference, while batch endpoints are used for offline batch scoring.

Commands

Name	Description	Type	Status
az ml online-endpoint create	Create an endpoint.	Extension	GA
az ml online-endpoint delete	Delete an endpoint.	Extension	GA
az ml online-endpoint get-credentials	List the token/keys for an online endpoint.	Extension	GA
az ml online-endpoint invoke	Invoke an endpoint.	Extension	GA
az ml online-endpoint list	List endpoints in a workspace.	Extension	GA
az ml online-endpoint regenerate-keys	Regenerate the keys for an online endpoint.	Extension	GA
az ml online-endpoint show	Show details for an endpoint.	Extension	GA
az ml online-endpoint update	Update an endpoint.	Extension	GA

az ml online-endpoint create

Create an endpoint.

To create an endpoint, provide a YAML file with online endpoint configuration. If the endpoint already exists, it will fail. If you want to update existing endpoint, use az ml online-endpoint update.

az ml online-endpoint create --resource-group
                             --workspace-name
                             [--auth-mode]
                             [--file]
                             [--local {false, true}]
                             [--name]
                             [--no-wait]
                             [--set]
                             [--web]

Examples

Create an endpoint from a YAML specification file

az ml online-endpoint create --file endpoint.yml --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.

Optional Parameters

The following parameters are optional, but depending on the context, one or more might become required for the command to execute successfully.

--auth-mode

Authentication method for the endpoint. Allowed values: key, aml_token. Default: key.

--file -f

Local path to the YAML file containing the Azure ML online-endpoint specification. The YAML reference docs for online-endpoint can be found at: https://aka.ms/ml-cli-v2-endpoint-online-yaml-reference.

--local

Create endpoint locally. Note: traffic and auth is not supported locally. You can use 'az ml online-deployment create --local' directly. It will create an endpoint if one doesn't exist.

Property	Value
Default value:	False
Accepted values:	false, true

--name -n

Name of the online endpoint.

--no-wait

Do not wait for the long-running-operation to finish. Default is False.

Property	Value
Default value:	False

--set

Update an object by specifying a property path and value to set. Example: --set property1.property2=value.

--web -e

Show the endpoint's details in Azure ML studio in a web browser.

Property	Value
Default value:	False

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

Property	Value
Default value:	False

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

Property	Value
Default value:	False

--output -o

Output format.

Property	Value
Default value:	json
Accepted values:	json, jsonc, none, table, tsv, yaml, yamlc

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

Property	Value
Default value:	False

az ml online-endpoint delete

Delete an endpoint.

az ml online-endpoint delete --name
                             --resource-group
                             --workspace-name
                             [--local {false, true}]
                             [--no-wait]
                             [--yes]

Examples

Delete an online endpoint, including all its deployments

az ml online-endpoint delete --name my-online-endpoint --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--name -n

Name of the online endpoint.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.

Optional Parameters

The following parameters are optional, but depending on the context, one or more might become required for the command to execute successfully.

--local

Delete local endpoint.

Property	Value
Default value:	False
Accepted values:	false, true

--no-wait

Do not wait for the long-running-operation to finish. Default is False.

Property	Value
Default value:	False

--yes -y

Do not prompt for confirmation.

Property	Value
Default value:	False

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

Property	Value
Default value:	False

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

Property	Value
Default value:	False

--output -o

Output format.

Property	Value
Default value:	json
Accepted values:	json, jsonc, none, table, tsv, yaml, yamlc

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

Property	Value
Default value:	False

az ml online-endpoint get-credentials

List the token/keys for an online endpoint.

az ml online-endpoint get-credentials --name
                                      --resource-group
                                      --workspace-name

Examples

List the keys for an online endpoint

az ml online-endpoint get-credentials --name my-online-endpoint --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--name -n

Name of the online endpoint.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

Property	Value
Default value:	False

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

Property	Value
Default value:	False

--output -o

Output format.

Property	Value
Default value:	json
Accepted values:	json, jsonc, none, table, tsv, yaml, yamlc

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

Property	Value
Default value:	False

az ml online-endpoint invoke

Invoke an endpoint.

You can invoke an online endpoint with some request data. This will be real-time inference, and the scoring results will be returned immediately.

az ml online-endpoint invoke --name
                             --resource-group
                             --workspace-name
                             [--deployment-name]
                             [--local {false, true}]
                             [--request-file]

Examples

Invoke an online endpoint with some request data

az ml online-endpoint invoke --name my-online-endpoint --request-file sample_request.json --resource-group my-resource-group --workspace-name my-workspace

Invoke an online endpoint, targeting a specific deployment

az ml online-endpoint invoke --name my-online-endpoint --deployment my-deployment --request-file sample_request.json --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--name -n

Name of the online endpoint.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.

Optional Parameters

The following parameters are optional, but depending on the context, one or more might become required for the command to execute successfully.

--deployment-name -d

Name of the deployment to target.

--local

Invoke local endpoint. This will only work if a local deployment has been created for this endpoint.

Property	Value
Default value:	False
Accepted values:	false, true

--request-file -r

Local path to the JSON file containing the request data.

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

Property	Value
Default value:	False

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

Property	Value
Default value:	False

--output -o

Output format.

Property	Value
Default value:	json
Accepted values:	json, jsonc, none, table, tsv, yaml, yamlc

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

Property	Value
Default value:	False

az ml online-endpoint list

List endpoints in a workspace.

az ml online-endpoint list --resource-group
                           --workspace-name
                           [--local {false, true}]

Examples

List all the online endpoints in a workspace

az ml online-endpoint list --resource-group my-resource-group --workspace-name my-workspace

List all the batch endpoints in a workspace

az ml online-endpoint list  --resource-group my-resource-group --workspace-name my-workspace

List all the online endpoints in a workspace using --query argument to execute a JMESPath query on the results of commands.

az ml online-endpoint list --query "[].{Name:name}"  --output table --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.

Optional Parameters

The following parameters are optional, but depending on the context, one or more might become required for the command to execute successfully.

--local

List all local endpoints.

Property	Value
Default value:	False
Accepted values:	false, true

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

Property	Value
Default value:	False

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

Property	Value
Default value:	False

--output -o

Output format.

Property	Value
Default value:	json
Accepted values:	json, jsonc, none, table, tsv, yaml, yamlc

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

Property	Value
Default value:	False

az ml online-endpoint regenerate-keys

Regenerate the keys for an online endpoint.

az ml online-endpoint regenerate-keys --name
                                      --resource-group
                                      --workspace-name
                                      [--key-type]
                                      [--no-wait]

Examples

Regenerate the keys for an online endpoint

az ml online-endpoint regenerate-keys --name my-online-endpoint --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--name -n

Name of the online endpoint.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.

Optional Parameters

The following parameters are optional, but depending on the context, one or more might become required for the command to execute successfully.

--key-type

The type of key to regenerate. Allowed values: primary, secondary.

Property	Value
Default value:	primary

--no-wait

Do not wait for the long-running-operation to finish. Default is False.

Property	Value
Default value:	False

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

Property	Value
Default value:	False

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

Property	Value
Default value:	False

--output -o

Output format.

Property	Value
Default value:	json
Accepted values:	json, jsonc, none, table, tsv, yaml, yamlc

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

Property	Value
Default value:	False

az ml online-endpoint show

Show details for an endpoint.

az ml online-endpoint show --name
                           --resource-group
                           --workspace-name
                           [--local {false, true}]
                           [--web]

Examples

Show the details for a batch endpoint

az ml online-endpoint show --name my-online-endpoint  --resource-group my-resource-group --workspace-name my-workspace

Show the provisioning state of an endpoint using --query argument to execute a JMESPath query on the results of commands.

az ml online-endpoint show -n my-endpoint --query "{Name:name,State:provisioning_state}"  --output table --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--name -n

Name of the online endpoint.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.

Optional Parameters

The following parameters are optional, but depending on the context, one or more might become required for the command to execute successfully.

--local

Show local endpoint.

Property	Value
Default value:	False
Accepted values:	false, true

--web -e

Show the endpoint's details in Azure ML studio in a web browser.

Property	Value
Default value:	False

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

Property	Value
Default value:	False

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

Property	Value
Default value:	False

--output -o

Output format.

Property	Value
Default value:	json
Accepted values:	json, jsonc, none, table, tsv, yaml, yamlc

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

Property	Value
Default value:	False

az ml online-endpoint update

Update an endpoint.

The 'description', 'tags', and 'traffic' properties of an endpoint can be updated. In addition, new deployments can be added to an endpoint, and existing deployments can be updated.

az ml online-endpoint update --resource-group
                             --workspace-name
                             [--add]
                             [--file]
                             [--force-string]
                             [--local {false, true}]
                             [--mirror-traffic]
                             [--name]
                             [--no-wait]
                             [--remove]
                             [--set]
                             [--traffic]
                             [--web]

Examples

Update an endpoint from a YAML specification file

az ml online-endpoint update --file updated_endpoint.yml --resource-group my-resource-group --workspace-name my-workspace

Update the traffic settings for an endpoint

az ml online-endpoint update --name my-online-endpoint  --traffic "my-new-deployment=100" --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default workspace using az configure --defaults workspace=<name>.

Optional Parameters

The following parameters are optional, but depending on the context, one or more might become required for the command to execute successfully.

--add

Add an object to a list of objects by specifying a path and key value pairs. Example: --add property.listProperty <key=value, string or JSON string>.

Property	Value
Parameter group:	Generic Update Arguments
Default value:	[]

--file -f

--force-string

When using 'set' or 'add', preserve string literals instead of attempting to convert to JSON.

Property	Value
Parameter group:	Generic Update Arguments
Default value:	False

--local

Update local endpoint.

Property	Value
Default value:	False
Accepted values:	false, true

--mirror-traffic

Directs a duplicate percentage of live traffic to a train a deployment.

--name -n

Name of the online endpoint.

--no-wait

Do not wait for the long-running-operation to finish. Default is False.

Property	Value
Default value:	False

--remove

Remove a property or an element from a list. Example: --remove property.list <indexToRemove> OR --remove propertyToRemove.

Property	Value
Parameter group:	Generic Update Arguments
Default value:	[]

--set

Update an object by specifying a property path and value to set. Example: --set property1.property2=<value>.

Property	Value
Parameter group:	Generic Update Arguments
Default value:	[]

--traffic -r

Space-separated key-value pairs, in quotes, for the traffic settings for the endpoint.

--web -e

Show the endpoint's details in Azure ML studio in a web browser.

Property	Value
Default value:	False

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

Property	Value
Default value:	False

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

Property	Value
Default value:	False

--output -o

Output format.

Property	Value
Default value:	json
Accepted values:	json, jsonc, none, table, tsv, yaml, yamlc

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

Property	Value
Default value:	False

Share via

az ml online-endpoint

Commands

az ml online-endpoint create

Examples

Required Parameters

Optional Parameters

az ml online-endpoint delete

Examples

Required Parameters

Optional Parameters

az ml online-endpoint get-credentials

Examples

Required Parameters

az ml online-endpoint invoke

Examples

Required Parameters

Optional Parameters

az ml online-endpoint list

Examples

Required Parameters

Optional Parameters

az ml online-endpoint regenerate-keys

Examples

Required Parameters

Optional Parameters

az ml online-endpoint show

Examples

Required Parameters

Optional Parameters

az ml online-endpoint update

Examples

Required Parameters

Optional Parameters

Feedback