az ml datastore

  • Reference

Note

This reference is part of the ml extension for the Azure CLI (version 2.15.0 or higher). The extension will automatically install the first time you run an az ml datastore command. Learn more about extensions.

Manage Azure ML datastores.

Azure ML datastores securely link your Azure storage services to your workspace so that you can access your storage without having to hardcode the connection information into your scripts. The connection secrets, like the storage service's authentication credentials, are stored in your workspace's Key Vault.

When you create a workspace, an Azure Storage account is automatically created as an associated resource. A blob container is created in this account, and its connection information is stored as a datastore named 'workspaceblobstore'. This serves as the workspace's default datastore, and the blob container is used to store your workspace artifacts and machine learning job logs and outputs.

Commands

az ml datastore create

Create a datastore.
az ml datastore delete

Delete a datastore.
az ml datastore list

List datastores in a workspace.
az ml datastore show

Show details for a datastore.
az ml datastore update

Update a datastore.

az ml datastore create

Create a datastore.

This connects the underlying Azure storage service to the workspace. The storage service types that can currently be connected to by creating a datastore include Azure Blob storage, Azure File Share, Azure Data Lake Storage Gen1 and Azure Data Lake Storage Gen2.

az ml datastore create --file
                       --resource-group
                       --workspace-name
                       [--name]
                       [--set]

Examples

Create a datastore from a YAML specification file

az ml datastore create --file blobstore.yml --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--file -f

Local path to the YAML file containing the Azure ML datastore specification. The YAML reference docs for datastore can be found at: https://aka.ms/ml-cli-v2-datastore-blob-yaml-reference, https://aka.ms/ml-cli-v2-datastore-file-yaml-reference, https://aka.ms/ml-cli-v2-datastore-data-lake-gen1-yaml-reference, https://aka.ms/ml-cli-v2-datastore-data-lake-gen2-yaml-reference.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--name -n

Name of the datastore. This overwrites the 'name' field in the YAML file provided to --file/-f.

--set

Update an object by specifying a property path and value to set. Example: --set property1.property2=.

az ml datastore delete

Delete a datastore.

This deletes the connection information to the storage service from the workspace but does not delete the underlying data in storage.

az ml datastore delete --name
                       --resource-group
                       --workspace-name

Required Parameters

--name -n

Name of the datastore.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

az ml datastore list

List datastores in a workspace.

az ml datastore list --resource-group
                     --workspace-name
                     [--max-results]

Examples

List all the datastores in a workspace using --query argument to execute a JMESPath query on the results of commands.

az ml datastore list --query "[].{Name:name}"  --output table --resource-group my-resource-group --workspace-name my-workspace

Required Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--max-results -r

Max number of results to return.

az ml datastore show

Show details for a datastore.

az ml datastore show --name
                     --resource-group
                     --workspace-name

Required Parameters

--name -n

Name of the datastore.

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

az ml datastore update

Update a datastore.

The 'description', 'tags', and 'credential' properties can be updated.

az ml datastore update --resource-group
                       --workspace-name
                       [--add]
                       [--file]
                       [--force-string]
                       [--name]
                       [--remove]
                       [--set]

Required Parameters

--resource-group -g

Name of resource group. You can configure the default group using az configure --defaults group=<name>.

--workspace-name -w

Name of the Azure ML workspace. You can configure the default group using az configure --defaults workspace=<name>.

Optional Parameters

--add

Add an object to a list of objects by specifying a path and key value pairs. Example: --add property.listProperty <key=value, string or JSON string>.

default value: []
--file -f

Local path to the YAML file containing the Azure ML datastore specification. The YAML reference docs for datastore can be found at: https://aka.ms/ml-cli-v2-datastore-blob-yaml-reference, https://aka.ms/ml-cli-v2-datastore-file-yaml-reference, https://aka.ms/ml-cli-v2-datastore-data-lake-gen1-yaml-reference, https://aka.ms/ml-cli-v2-datastore-data-lake-gen2-yaml-reference.

--force-string

When using 'set' or 'add', preserve string literals instead of attempting to convert to JSON.

default value: False
--name -n

Name of the datastore. This overwrites the 'name' field in the YAML file provided to --file/-f.

--remove

Remove a property or an element from a list. Example: --remove property.list OR --remove propertyToRemove.

default value: []
--set

Update an object by specifying a property path and value to set. Example: --set property1.property2=.

default value: []