AmlCompute Class

  • Reference

Manages an Azure Machine Learning compute in Azure Machine Learning.

An Azure Machine Learning Compute (AmlCompute) is a managed-compute infrastructure that allows you to easily create a single or multi-node compute. The compute is created within your workspace region as a resource that can be shared with other users. For more information, see What are compute targets in Azure Machine Learning?

Class ComputeTarget constructor.

Retrieve a cloud representation of a Compute object associated with the provided workspace. Returns an instance of a child class corresponding to the specific type of the retrieved Compute object.

Inheritance
ComputeTarget
AmlCompute

Constructor

AmlCompute(workspace, name)

Parameters

Name Description
workspace
Required
Workspace

The workspace object containing the AmlCompute object to retrieve.
name
Required
str

The name of the AmlCompute object to retrieve.
workspace
Required
Workspace

The workspace object containing the Compute object to retrieve.
name
Required
str

The name of the of the Compute object to retrieve.

Remarks

In the following example, a persistent compute target provisioned by AmlCompute is created. The provisioning_configuration parameter in this example is of type AmlComputeProvisioningConfiguration, which is a child class of ComputeTargetProvisioningConfiguration.


   from azureml.core.compute import ComputeTarget, AmlCompute
   from azureml.core.compute_target import ComputeTargetException

   # Choose a name for your CPU cluster
   cpu_cluster_name = "cpu-cluster"

   # Verify that cluster does not exist already
   try:
       cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)
       print('Found existing cluster, use it.')
   except ComputeTargetException:
       compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_D2_V2',
                                                              max_nodes=4)
       cpu_cluster = ComputeTarget.create(ws, cpu_cluster_name, compute_config)

   cpu_cluster.wait_for_completion(show_output=True)

Full sample is available from https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/training/train-on-amlcompute/train-on-amlcompute.ipynb

Methods

add_identity

Add Identity Type and/or Identity Ids for this AmlCompute target.
delete

Remove the AmlCompute object from its associated workspace.
deserialize

Convert a JSON object into an AmlCompute object.
detach

Detach is not supported for AmlCompute object. Use delete instead.
get

Return compute object.
get_active_runs

Return a generator of the runs for this compute.
get_status

Retrieve the current detailed status for the AmlCompute cluster.
list_nodes

Get the details (e.g., IP address, port, etc.) of all the compute nodes in the compute target.
list_quotas

Get the currently assigned Workspace quotas based on VMFamily for given workspace and subscription.
list_usages

Get the current usage information as well as limits for AML resources for given workspace and subscription.
provisioning_configuration

Create a configuration object for provisioning an AmlCompute target.
refresh_state

Perform an in-place update of the properties of the object.

This method updates the properties based on the current state of the corresponding cloud object. This is primarily used for manual polling of compute state.
remove_identity

Remove the identity on the compute.
serialize

Convert this AmlCompute object into a JSON serialized dictionary.
supported_vmsizes

List the supported VM sizes in a region.
update

Update the ScaleSettings for this AmlCompute target.
update_quotas

Update quota for a VM family in workspace.
wait_for_completion

Wait for the AmlCompute cluster to finish provisioning.

This can be configured to wait for a minimum number of nodes, and to timeout after a set period of time.

add_identity

Add Identity Type and/or Identity Ids for this AmlCompute target.

add_identity(identity_type, identity_id=None)

Parameters

Name Description
identity_type
Required
string

Possible values are:

  • SystemAssigned - System assigned identity

  • UserAssigned - User assigned identity. Requires identity id to be set.
identity_id
list[str]

List of resource ids for the user assigned identity. eg. ['/subscriptions//resourceGroups//providers/Microsoft.ManagedIdentity /userAssignedIdentities/']

default value: None

Remarks

identity_id should only be specified when identity_type == UserAssigned

delete

Remove the AmlCompute object from its associated workspace.

delete()

Exceptions

Type Description
ComputeTargetException

Remarks

If this object was created through Azure Machine Learning, the corresponding cloud-based objects will also be deleted. If this object was created externally and only attached to the workspace, this method raises a ComputeTargetException and nothing is changed.

deserialize

Convert a JSON object into an AmlCompute object.

static deserialize(workspace, object_dict)

Parameters

Name Description
workspace
Required
Workspace

The workspace object the AmlCompute object is associated with.
object_dict
Required
dict

A JSON object to convert to an AmlCompute object.

Returns

Type Description
AmlCompute

The AmlCompute representation of the provided JSON object.

Exceptions

Type Description
ComputeTargetException

Remarks

Raises a ComputeTargetException if the provided workspace is not the workspace the Compute is associated with.

detach

Detach is not supported for AmlCompute object. Use delete instead.

detach()

Exceptions

Type Description
ComputeTargetException

get

Return compute object.

get()

get_active_runs

Return a generator of the runs for this compute.

get_active_runs(type=None, tags=None, properties=None, status=None)

Parameters

Name Description
type
str

Filter the returned generator of runs by the provided type. See add_type_provider for creating run types.

default value: None
tags
str or dict

Filter runs by "tag" or {"tag": "value"}

default value: None
properties
str or dict

Filter runs by "property" or {"property": "value"}

default value: None
status
str

Run status - either "Running" or "Queued"

default value: None

Returns

Type Description
<xref:builtin.generator>

a generator of ~_restclient.models.RunDto

get_status

Retrieve the current detailed status for the AmlCompute cluster.

get_status()

Returns

Type Description
AmlComputeStatus

A detailed status object for the cluster

list_nodes

Get the details (e.g., IP address, port, etc.) of all the compute nodes in the compute target.

list_nodes()

Returns

Type Description
list

The details of all the compute nodes in the compute target.

list_quotas

Get the currently assigned Workspace quotas based on VMFamily for given workspace and subscription.

static list_quotas(workspace, location=None)

Parameters

Name Description
workspace
Required
Workspace
location
str

The location of the quotas. If not specified, will default to workspace location.

default value: None

Returns

Type Description
list

List of currently assigned Workspace Quotas based on VMFamily

list_usages

Get the current usage information as well as limits for AML resources for given workspace and subscription.

static list_usages(workspace, show_all=False, location=None)

Parameters

Name Description
workspace
Required
Workspace
show_all

Specifies if detailed usages of child resources are required. Defaults to False

default value: False
location
str

The location of the resources. If not specified, will default to workspace location.

default value: None

Returns

Type Description
list

List of current usage information as well as limits for AML resources

provisioning_configuration

Create a configuration object for provisioning an AmlCompute target.

static provisioning_configuration(vm_size='', vm_priority='dedicated', min_nodes=0, max_nodes=None, idle_seconds_before_scaledown=1800, admin_username=None, admin_user_password=None, admin_user_ssh_key=None, vnet_resourcegroup_name=None, vnet_name=None, subnet_name=None, tags=None, description=None, remote_login_port_public_access='NotSpecified', identity_type=None, identity_id=None, location=None, enable_node_public_ip=True)

Parameters

Name Description
vm_size
Required
str

The size of agent VMs. More details can be found here: https://aka.ms/azureml-vm-details. Note that not all sizes are available in all regions, as detailed in the previous link. If not specified, defaults to Standard_NC6.
vm_priority
str

The VM priority, dedicated or lowpriority.

default value: dedicated
min_nodes
int

The minimum number of nodes to use on the cluster. If not specified, defaults to 0.

default value: 0
max_nodes
int

The maximum number of nodes to use on the cluster. If not specified, defaults to 4.

default value: None
idle_seconds_before_scaledown
int

Node idle time in seconds before scaling down the cluster. If not specified, defaults to 1800.

default value: 1800
admin_username
str

The name of the administrator user account which can be used to SSH into nodes.

default value: None
admin_user_password
str

The password of the administrator user account.

default value: None
admin_user_ssh_key
str

The SSH public key of the administrator user account.

default value: None
vnet_resourcegroup_name
str

The name of the resource group where the virtual network is located.

default value: None
vnet_name
str

The name of the virtual network.

default value: None
subnet_name
str

The name of the subnet inside the VNet.

default value: None
tags
dict[str, str]

A dictionary of key value tags to provide to the compute object.

default value: None
description
str

A description to provide to the compute object.

default value: None
remote_login_port_public_access
str

State of the public SSH port. Possible values are:

  • Disabled - Indicates that the public ssh port is closed on all nodes of the cluster.

  • Enabled - Indicates that the public ssh port is open on all nodes of the cluster.

  • NotSpecified - Indicates that the public ssh port is closed on all nodes of the cluster if VNet is defined, else is open all public nodes. It can be this default value only during cluster creation time. After creation, it will be either enabled or disabled.

default value: NotSpecified
identity_type
string

Possible values are:

  • SystemAssigned - System assigned identity

  • UserAssigned - User assigned identity. Requires identity id to be set.

default value: None
identity_id
list[str]

List of resource ids for the user assigned identity. eg. ['/subscriptions//resourceGroups//providers/Microsoft.ManagedIdentity/userAssignedIdentities/']

default value: None
location
str

Location to provision cluster in.

default value: None
enable_node_public_ip
bool

Enable node public IP. Possible values are:

  • True - Enable node public IP.

  • False - Disable node public IP.

  • NotSpecified - Enable node public IP.

default value: True

Returns

Type Description
AmlComputeProvisioningConfiguration

A configuration object to be used when creating a Compute object.

Exceptions

Type Description
ComputeTargetException

refresh_state

Perform an in-place update of the properties of the object.

This method updates the properties based on the current state of the corresponding cloud object. This is primarily used for manual polling of compute state.

refresh_state()

remove_identity

Remove the identity on the compute.

remove_identity(identity_id=None)

Parameters

Name Description
identity_id
list[str]

User assigned identities

default value: None

Remarks

System Assigned identity will be removed automatically if identity_id is not specified

serialize

Convert this AmlCompute object into a JSON serialized dictionary.

serialize()

Returns

Type Description
dict

The JSON representation of this AmlCompute object.

supported_vmsizes

List the supported VM sizes in a region.

static supported_vmsizes(workspace, location=None)

Parameters

Name Description
workspace
Required
Workspace
location
str

The location of cluster. If not specified, will default to workspace location.

default value: None

Returns

Type Description
list

A list of supported VM sizes in a region with names of the VM, VCPUs, and RAM.

update

Update the ScaleSettings for this AmlCompute target.

update(min_nodes=None, max_nodes=None, idle_seconds_before_scaledown=None)

Parameters

Name Description
min_nodes
int

The minimum number of nodes to use on the cluster.

default value: None
max_nodes
int

The maximum number of nodes to use on the cluster.

default value: None
idle_seconds_before_scaledown
int

The node idle time in seconds before scaling down the cluster.

default value: None

update_quotas

Update quota for a VM family in workspace.

static update_quotas(workspace, vm_family, limit=None, location=None)

Parameters

Name Description
workspace
Required
Workspace
vm_family
Required
str

VM family name
limit
int

The maximum permitted quota of the resource

default value: None
location
str

The location of the quota. If not specified, will default to workspace location.

default value: None

wait_for_completion

Wait for the AmlCompute cluster to finish provisioning.

This can be configured to wait for a minimum number of nodes, and to timeout after a set period of time.

wait_for_completion(show_output=False, min_node_count=None, timeout_in_minutes=25, is_delete_operation=False)

Parameters

Name Description
show_output
bool

Boolean to provide more verbose output.

default value: False
min_node_count
int

Minimum number of nodes to wait for before considering provisioning to be complete. This doesn't have to equal the minimum number of nodes that the compute was provisioned with, however it should not be greater than that.

default value: None
timeout_in_minutes
int

The duration in minutes to wait before considering provisioning to have failed.

default value: 25
is_delete_operation
bool

Indicates whether the operation is meant for deleting.

default value: False

Exceptions

Type Description
ComputeTargetException