Registered Dataset is not logged as reference in Azure Machine Service Experiment

Hamsini S 1

Hi there,

I have tried executing the notebook "tutorial-1st-experiment-sdk-train.ipynb" given by Azure as part of the tutorial in Azure Machine Learning service itself.

The only change I made was, rather taking the input directly from package, I registered the dataset first and referred that in this notebook. My intention was to see how input dataset can be tracked in the experiment.

But, no luck. I was able to successfully execute the notebook, but in experiment, "Input datasets" was blank, not referenced, though I used a registered latest version of dataset from Azure as my input.

How do we tag the input dataset to an experiment when executing from Azure Machine Learning Notebook?

Thanks,

GiftA-MSFT 11,161

Hi, thanks for reaching out. To access your registered dataset across experiments, use the following code to access your workspace and registered dataset by name:

%%writefile $script_folder/train.py  
  
from azureml.core import Dataset, Run  
  
run = Run.get_context()  
workspace = run.experiment.workspace  
  
dataset_name = 'titanic_ds'  
  
# Get a dataset by name  
titanic_ds = Dataset.get_by_name(workspace=workspace, name=dataset_name)  
  
# Load a TabularDataset into pandas DataFrame  
df = titanic_ds.to_pandas_dataframe()

Let me know if the above response helped, otherwise, feel free to clarify so we can assist further. Thanks.

Hamsini S 1 Reputation point

2020-10-01T15:38:16.353+00:00

Thanks for your response. But my question was more on how to tag the input dataset that is registered in Azure to an experiment while executing that experiment from Azure Machine Learning notebook.

See in the screenshot, how do I set "input dataset" which would take me to registered dataset in Azure Machine Learning.

Thanks,
Hamsini S 1 Reputation point

2020-10-01T16:33:40.393+00:00

It looks like the attachment did not go right. Attaching it again.
GiftA-MSFT 11,161 Reputation points

2020-10-01T22:17:37.84+00:00
To get the registered dataset from your workspace, you can use the following code. However, can you please share the code you used when referencing the dataset (i.e the code you added to the sample tutorial) to help clarify the problem more? Thanks.

x_df = Dataset.get_by_name(ws, name='dataset_name')
Hamsini S 1 Reputation point

2020-10-05T10:01:55.537+00:00

Hi there,

Here is my code to refer the registered dataset

from azureml.core import Dataset, Experiment
from azureml.core import Workspace
ws = Workspace.from_config()
input_dataset = Dataset.get_by_name(workspace = ws, name = <name>, version = 'latest')

My question is not about referring registered dataset in the notebook. But, my question is, how to get the reference of input registered dataset in "Experiment" menu. Refer the attachment where I have highlighted the section where I want to see the input dataset reference from registered section.

Thanks and Regards,
GiftA-MSFT 11,161 Reputation points

2020-10-08T18:48:05.677+00:00

Thanks for following up and sorry for the delayed response. From my understanding, the input dataset should automatically link to your experiment. However, I was able to reproduce the same behavior you observed. We are reviewing this issue internally and will update you soon with our findings. Thanks.
GiftA-MSFT 11,161 Reputation points

2020-10-22T18:50:13.67+00:00

A quick follow-up. This issue has been identified as a bug and a fix is underway. Will share updates accordingly. Thanks for your patience.

Share via

Registered Dataset is not logged as reference in Azure Machine Service Experiment