confusion about the containers name in Azure storage account and Azure machine learning

Question

I feel very confused about the container name in Azure storage account and Azure machine learning account.

I upload a data in the container in Azure blob storage account called "azureml"

But when I create a data asset for my Azure machine learning. Its name becomes "workspaceartifactstore" because I find the file I uploaded is located in it. It is very confusing.

Answer 1

Hi Yu Cai,

Thanks for the interesting question. I would let you know few details about what is what here, It seems like there might be a misunderstanding or misconfiguration in your Azure Machine Learning workspace. . I would clarify the concepts and provide guidance on how to handle containers in Azure Storage and Azure Machine Learning.

In Azure blob storage, containers are used to organize sets of blobs. When you upload data to a container in Azure Blob Storage, you are essentially storing files in that containers.

In Azure Machine Learning, the concept of a "datastore" is used to reference storage locations for your data. The default datastore associated with an Azure Machine Learning workspace is often named "workspaceblobstore." Within this default datastore, a container named "azureml" is commonly used to store artifacts related to the Azure Machine Learning workspace.

This is where the major confusion for you resides I believe.

There are few links where we can have the references about the naming conventions:

Link 1: https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ready/azure-best-practices/resource-naming

Link 2: https://learn.microsoft.com/en-us/rest/api/storageservices/naming-and-referencing-containers--blobs--and-metadata

Regarding this specific situation, if you uploaded data to a container named "azureml" in Azure Blob Storage, it should not be directly related to the "workspaceartifactstore" container. The "workspaceartifactstore" container is typically created automatically within the "workspaceblobstore" datastore and is used to store various artifacts related to the Azure Machine Learning workspace.

If you want to use the data you uploaded to the "azureml" container in your Azure Machine Learning workspace, you should create a datastore in your workspace that points to the "azureml" container in your Azure Blob Storage account.

Here is the reference link to create the datastore in the workspace: https://learn.microsoft.com/en-us/cli/azure/ml/datastore?view=azure-cli-latest

Good luck with your exploration and development efforts, and feel free to reach out through this comment if you need further assistance.

Best regards,

Chakravarthi Rangarajan Bhargavi

-Please kindly accept the answer and vote 'Yes' if you feel helpful to support the community, thanks a lot.

Answer 2

Thank you, Chakravarthi, I will study the code and update your feedback. Thank for Azure team support. It takes time for me to learn Azure.

Answer 3

Hello Chakravarthi, I check your instructions: "Regarding this specific situation, if you uploaded data to a container named "azureml" in Azure Blob Storage, it should not be directly related to the "workspaceartifactstore" container. The "workspaceartifactstore" container is typically created automatically within the "workspaceblobstore" datastore and is used to store various artifacts related to the Azure Machine Learning workspace.",

That is my problem. I upload the .csv file to "azureml" container in my Azure storage account with a directory "ExperimentRun". I found the csv data in "workspaceartifactstore" container instead of "workspaceblobstore" container in Azure ML Datastores. I feel very confused.

Answer 4

Hello I checked my workspaceblobstore(default), I find it has a problem. This is created by default when I create Azure ML service. But the container "workspaceartifactstore" is accessible. Do you have any suggestions? I feel very confused.