Azure Machine Learning
An Azure machine learning service for building and deploying models.
1,869 questions
Azure ML - Can not list/load registered Datasets via Python SDK
Lukas
121
Reputation points
I'm quite new to Azure ML and Python. I created some datasets using both the Azure ML GUI and the Python SDK:
Now I want to load these datasets in a Pandas Dataframe. But when I run
Dataset.get_all(workspace=workspace)
I got an empty list:
Do I miss something? I'm using the version 0.2.7. of azureml and Version 1.46.0. of azureml-core.
I also tried
workspace.datasets
But also got an empty result.
{count} vote
-
romungi-MSFT 32,686 Reputation points Microsoft Employee2022-11-09T12:01:13.49+00:00 @Lukas Are you setting the workspace object with the subscription id, resource group and workspace name?
subscription_id = '<subscription_id>' resource_group = '<resource_group>' workspace_name = '<workspace_name>' workspace1 = Workspace(subscription_id, resource_group, workspace_name)If Yes, This should print the details of registered datasets. You could also try to print the result and check if it works?
print(Dataset.get_all(workspace=workspace1)Loading them in a dataframe works fine for me.
-
Lukas 121 Reputation points
2022-11-09T12:44:08.2+00:00 Yes Sir, I do:
Moreover I can see the datasets in the pipeline desinger:
I use the following tutorial to create the worksapce and all necessary ressources: https://learn.microsoft.com/en-us/azure/machine-learning/tutorial-create-secure-workspace
Maybe there are network settings which I'm missing?
-
romungi-MSFT 32,686 Reputation points Microsoft Employee
2022-11-09T16:54:30.307+00:00 I think this could be the reason since you have created a workspace behind a VNET. The documentation in this section mentions that Jupyter, JupyterLab, and RStudio on a compute instance, including running notebooks, is not supported. You could probably set
public_network_accessto true from Azure portal as recommended.If you do not intend to enable this, try using an Azure Bastion and install the SDK on the bastion host to run the commands.
-
Lukas 121 Reputation points
2022-11-10T07:52:17.14+00:00 No Sir! After enabling the network access from all networks, it is still not working:
Of course I waited a few minutes after I changed the firewall rules.
-
romungi-MSFT 32,686 Reputation points Microsoft Employee
2022-11-10T12:38:43.14+00:00 @Lukas Did you also try using the bastion host and using the SDK outside of Jupyter notebook of the Azure ML studio?
I do not have a similar network setup to replicate the scenario. If this is blocking your work, you can raise an azure support case to review your setup with a service engineer. Thanks!! -
Hillaker, Annie 1 Reputation point
2022-12-24T23:40:33.157+00:00 Any solution with this. I had the exact same issues. Even I turned on the public access to workspace and data storage, it still has the same issues.
Sign in to comment