SynapseSparkCompute Class

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

SynapseSpark Compute resource.

Inheritance

azure.ai.ml.entities._compute.compute.Compute

SynapseSparkCompute

Constructor

SynapseSparkCompute(*, name: str, description: str | None = None, tags: Dict[str, str] | None = None, node_count: int | None = None, node_family: str | None = None, node_size: str | None = None, spark_version: str | None = None, identity: IdentityConfiguration | None = None, scale_settings: AutoScaleSettings | None = None, auto_pause_settings: AutoPauseSettings | None = None, **kwargs: Any)

Keyword-Only Parameters

Name	Description
name	str The name of the compute.
description	Optional[str] The description of the resource. Defaults to None.
tags	Optional[[dict[str, str]] The set of resource tags defined as key/value pairs. Defaults to None.
node_count	Optional[int] The number of nodes in the compute.
node_family	Optional[str] The node family of the compute.
node_size	Optional[str] The size of the node.
spark_version	Optional[str] The version of Spark to use.
identity	Optional[IdentityConfiguration] The configuration of identities that are associated with the compute cluster.
scale_settings	Optional[AutoScaleSettings] The scale settings for the compute.
auto_pause_settings	Optional[AutoPauseSettings] The auto pause settings for the compute.
kwargs	Optional[dict] Additional keyword arguments passed to the parent class.

Examples

Creating Synapse Spark compute.

   from azure.ai.ml.entities import (
       AutoPauseSettings,
       AutoScaleSettings,
       IdentityConfiguration,
       ManagedIdentityConfiguration,
       SynapseSparkCompute,
   )

   synapse_compute = SynapseSparkCompute(
       name="synapse_name",
       resource_id="/subscriptions/subscription/resourceGroups/group/providers/Microsoft.Synapse/workspaces/workspace/bigDataPools/pool",
       identity=IdentityConfiguration(
           type="UserAssigned",
           user_assigned_identities=[
               ManagedIdentityConfiguration(
                   resource_id="/subscriptions/subscription/resourceGroups/group/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identity"
               )
           ],
       ),
       scale_settings=AutoScaleSettings(min_node_count=1, max_node_count=3, enabled=True),
       auto_pause_settings=AutoPauseSettings(delay_in_minutes=10, enabled=True),
   )

Methods

dump	Dump the compute content into a file in yaml format.

dump

Dump the compute content into a file in yaml format.

dump(dest: str | PathLike | IO, **kwargs: Any) -> None

Parameters

Name	Description
dest Required	Union[<xref:PathLike>, str, IO[AnyStr]] The destination to receive this compute's content. Must be either a path to a local file, or an already-open file stream. If dest is a file path, a new file will be created, and an exception is raised if the file exists. If dest is an open file, the file will be written to directly, and an exception will be raised if the file is not writable.'.

Attributes

base_path

The base path of the resource.

Returns

Type	Description
str	The base path of the resource.

created_on

The compute resource creation timestamp.

Returns

Type	Description
Optional[str]	The compute resource creation timestamp.

creation_context

The creation context of the resource.

Returns

Type	Description
Optional[SystemData]	The creation metadata for the resource.

id

The resource ID.

Returns

Type	Description
Optional[str]	The global ID of the resource, an Azure Resource Manager (ARM) ID.

provisioning_errors

The compute resource provisioning errors.

Returns

Type	Description
Optional[str]	The compute resource provisioning errors.

provisioning_state

The compute resource's provisioning state.

Returns

Type	Description
Optional[str]	The compute resource's provisioning state.

type

The compute type.

Returns

Type	Description
Optional[str]	The compute type.