AzureBlobDatastore Class

Represents a datastore that saves connection information to Azure Blob storage.

You should not work with this class directly. To create a datastore of this type, use the register_azure_blob_container method of Datastore.

Note: When using a datastore to access data, you must have permission to access that data, which depends on the credentials registered with the datastore.

Initialize a new Azure Blob Datastore.

Inheritance

AbstractAzureStorageDatastore

AzureBlobDatastore

Constructor

AzureBlobDatastore(workspace, name, container_name, account_name, sas_token=None, account_key=None, protocol=None, endpoint=None, request_session=None, workspace_msi_has_access=False, subscription_id=None, resource_group=None, service_data_access_auth_identity=None)

Parameters

Name	Description
workspace Required	Workspace The workspace this datastore belongs to.
name Required	str The name of the datastore. It can only contain alphanumeric characters or - or _.
container_name Required	str The blob container name.
account_name Required	str The storage account name.
sas_token	str, <xref:optional> The SAS token for accessing this container. Defaults to None. default value: None
account_key	str, <xref:optional> The storage account key. Defaults to None. default value: None
protocol	str, <xref:optional> The protocol to use to connect to the storage account. If None, defaults to https. default value: None
endpoint	str, <xref:optional> The endpoint of the blob container. If None, defaults to core.windows.net. default value: None
request_session	<xref:requests.Session>, <xref:optional> the session object to use for http requests. Defaults to None. default value: None
workspace Required	Workspace The workspace this datastore belongs to.
name Required	str The name of the datastore. It can only contain alphanumeric characters or - or _.
container_name Required	str The blob container name.
account_name Required	str The storage account name.
sas_token Required	str, <xref:optional> The SAS token for accessing this container. Defaults to None.
account_key Required	str, <xref:optional> The storage account key. Defaults to None.
protocol Required	str, <xref:optional> The protocol to use to connect to the storage account. If None, defaults to https.
endpoint Required	str, <xref:optional> The endpoint of the blob container. If None, defaults to core.windows.net.
request_session Required	<xref:requests.Session>, <xref:optional> The session object to use for http requests. Defaults to None.
workspace_msi_has_access	bool, <xref:optional> Whether the Workspace MSI has access to this storage account.(Deprecated) default value: False
subscription_id	str, <xref:optional> The subscription ID of the storage account. default value: None
resource_group	str, <xref:optional> The resource group of the storage account. default value: None
service_data_access_auth_identity	str or <xref:_restclient.models.ServiceDataAccessAuthIdentity> Indicates which identity to use to authenticate service data access to customer's storage. Possible values include: 'None', 'WorkspaceSystemAssignedIdentity', 'WorkspaceUserAssignedIdentity' default value: None

Methods

download	Download the data from the blob container to the local file system.
upload	Upload the data from the local file system to blob container this data store points to. Note: For this method to work, the datastore object must have either account key or SAS token property populated. If datastore object doesn't contain any form of credential, this method will fail. Moreover, this method has been deprecated, may be removed in future. We suggest that you use the Dataset.File.upload_directory method to upload your files from local directory and create FileDataset in a single method call.
upload_files	Upload the data from the local file system to the blob container this datastore points to. Note: This method is deprecated and will no longer be supported. Recommend to use FileDatasetFactory.upload_directory instead.

download

Download the data from the blob container to the local file system.

download(target_path, prefix=None, overwrite=False, show_progress=True)

Parameters

Name	Description
target_path Required	str The local directory to download the file to.
prefix	str, <xref:optional> The path to the folder in the blob container to download. If set to None, will download everything in the blob. Defaults to None. default value: None
overwrite	bool, <xref:optional> Indicates whether to overwrite existing files. Defaults to False. default value: False
show_progress	bool, <xref:optional> Indicates whether to show the progress of the download in the console. Defaults to True. default value: True

Returns

Type	Description
int	The number of files successfully downloaded.

upload

Upload the data from the local file system to blob container this data store points to.

Note: For this method to work, the datastore object must have either account key or SAS token property populated. If datastore object doesn't contain any form of credential, this method will fail. Moreover, this method has been deprecated, may be removed in future. We suggest that you use the Dataset.File.upload_directory method to upload your files from local directory and create FileDataset in a single method call.

upload(src_dir, target_path=None, overwrite=False, show_progress=True)

Parameters

Name	Description
src_dir Required	str The local directory to upload.
target_path Required	str The location in blob container to upload to. If None, then upload to root. Defaults to None.
overwrite Required	bool, <xref:optional> Indicates whether to overwrite existing files. Defaults to False.
show_progress Required	bool, <xref:optional> Indicates whether to show progress of the upload in the console. Defaults to True.

Returns

Type	Description
DataReference	The DataReference instance for the target path uploaded.

upload_files

Upload the data from the local file system to the blob container this datastore points to.

Note: This method is deprecated and will no longer be supported. Recommend to use FileDatasetFactory.upload_directory instead.

upload_files(files, relative_root=None, target_path=None, overwrite=False, show_progress=True)

Parameters

Name	Description
files Required	list[str] A list of absolute paths of files to upload.
relative_root Required	str, <xref:optional> The root used to determine the path of the files in the blob. For example, if we upload /path/to/file.txt, and we define base path to be /path, when file.txt is uploaded to the blob storage, it will have the path of /to/file.txt. If target_path is also given, then it will be used as the prefix for the derived path from above. The base path must be a common path of all of the files, otherwise an exception will be thrown. Defaults to None, which will find the common path.
target_path Required	str, <xref:optional> The location in the blob container to upload the data to. Defaults to None, the root.
overwrite Required	bool, <xref:optional> Indicates whether to overwrite existing files. Defaults to False.
show_progress Required	bool, <xref:optional> Indicates whether to show progress of the upload in the console. Defaults to True.

Returns

Type	Description
DataReference	The DataReference instance for the target path uploaded.