How to get the path of data loaded in a datastore for yolo training in azure machine learning studio

Question

How to get the path of data loaded in a datastore for yolo training in azure machine learning studio

Nazifi Peyman 0

I created a data assets as yoloasset is conneted to my data store as yolostorefolder which is connected to my images in the yoloimages container of my blob storage container and the images and labels are under the following folder structure in the container of the blob storage:

datasets/train/images/
datasets/train/labels/
datasets/val/images/
datasets/val/labels/

now to be able to run a YOLO training I need to enter the paths for train and val folders in a config file in yolo, how can I get those paths to do a yolo training in a jupyter notebood with my azure machine learning subscription?

B santhiswaroop naik 405 Reputation points

2023-07-04T18:11:14.0833333+00:00

To get the paths for the train and validation folders in your Azure Machine Learning (AML) workspace, you can use the AML SDK and Azure Blob Storage Python SDK. Here's an example of how you can retrieve the paths in a Jupyter notebook:

Python coding

Import the required libraries:

from azureml.core import Workspace, Datastore, Dataset

Connect to your AML workspace:

Load your AML workspace

ws = Workspace.from_config()

Retrieve the data asset and data store:

Get the data asset

data_asset = Dataset.get_by_name(workspace=ws, name='yoloasset')

Get the data store and container

datastore = Datastore.get(ws, 'yolostorefolder')

container_name = 'yoloimages'

Retrieve the paths for the train and validation folders:

Get the paths for the train and validation folders

train_folder_path = datastore.path(f'{container_name}/datasets/train').as_mount()

val_folder_path = datastore.path(f'{container_name}/datasets/val').as_mount()

Now you can use the train_folder_path and val_folder_path variables to specify the paths in your YOLO training configuration file.

Note: The code assumes that you have already created a connection to your AML workspace and have the necessary permissions to access the data asset and data store. Adjust the code accordingly if you need to authenticate or provide additional configuration.

By using the AML SDK and the Azure Blob Storage Python SDK, you can retrieve the paths to your data in Azure Blob Storage and utilize them in your YOLO training process within a Jupyter notebook in your AML subscription.

Make sure you have the necessary dependencies installed, such as the azureml-core and azure-storage-blob packages. You can install them using pip install azureml-core azure-storage-blob.

Your answer

B santhiswaroop naik 405 Reputation points

2023-07-04T18:11:14.0833333+00:00

To get the paths for the train and validation folders in your Azure Machine Learning (AML) workspace, you can use the AML SDK and Azure Blob Storage Python SDK. Here's an example of how you can retrieve the paths in a Jupyter notebook:

Python coding

Import the required libraries:

from azureml.core import Workspace, Datastore, Dataset

Connect to your AML workspace:

Load your AML workspace

ws = Workspace.from_config()

Retrieve the data asset and data store:

Get the data asset

data_asset = Dataset.get_by_name(workspace=ws, name='yoloasset')

Get the data store and container

datastore = Datastore.get(ws, 'yolostorefolder')

container_name = 'yoloimages'

Retrieve the paths for the train and validation folders:

Get the paths for the train and validation folders

train_folder_path = datastore.path(f'{container_name}/datasets/train').as_mount()

val_folder_path = datastore.path(f'{container_name}/datasets/val').as_mount()

Now you can use the train_folder_path and val_folder_path variables to specify the paths in your YOLO training configuration file.

Note: The code assumes that you have already created a connection to your AML workspace and have the necessary permissions to access the data asset and data store. Adjust the code accordingly if you need to authenticate or provide additional configuration.

By using the AML SDK and the Azure Blob Storage Python SDK, you can retrieve the paths to your data in Azure Blob Storage and utilize them in your YOLO training process within a Jupyter notebook in your AML subscription.

Make sure you have the necessary dependencies installed, such as the azureml-core and azure-storage-blob packages. You can install them using pip install azureml-core azure-storage-blob.

Share via

How to get the path of data loaded in a datastore for yolo training in azure machine learning studio

Load your AML workspace

Get the data asset

Get the data store and container

Get the paths for the train and validation folders

Your answer