How to use custom environment defined in the custom environments tab in Azure Machine Learning Studio.

Question

In Azure Machine Learning Studio, in the Environments section's "Custom environments" tab, I defined a custom environment. I have done this once with a Conda yaml file, and once with a requirements.txt file, eg filling out this form as described here: https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-environments-in-studio

I can see that the environments have been created but I have no idea how to use them.

I have tried using this code within an Azure ML Studio notebook, where the new environment I defined is called "my_new_env":

from azureml.core import Workspace, Environment  
ws = Workspace.from_config()  
env = Environment.get(workspace=ws, name="my_new_env")  

I also tried this within an Azure ML Studio notebook to see if I could define an environment without doing it in the environments menu.

from azureml.core.environment import Environment  
my_new_env = Environment.from_conda_specification(name = "myenv", file_path = environment.yml)  

Both execute without any warnings or errors, but I'm not sure they are running, or indeed what I have done.

Having run either of these two blocks of code, when I try to select a new environment in the Azure ML Studio notebook's drop down menu:

There is no evidence of my new environments.

I'm new to Azure ML Studio. What I want to do is create a new Python virtual environment. Am I getting confused between virtual Python environments and some other more general type of environments? If I wanted to create my own stable Python virtual environment within Azure ML to use in notebooks that was not tied to a specific compute instance, how would I do it?

Thanks!

Answer 1

@ml In the first two cases the environments created from portal or the yaml file are used to train your model or score your model when it is deployed as a endpoint.
These are re-usable environments that can even be created on local machine or compute to develop your training script and can be used used on Azure compute for large scale training or deployment.

The different types of environments are curated, user-managed and system-managed. The first two environments you created fall under the user and system managed categories, whereas curated environments are offered by azure and are available by default in every workspace.

These environments are not tied to any of the compute instance and you can use them with any type of compute for training or inference.

The last type of environment that you have listed with the screen shot is actually a virtual environment that you have to setup on your compute instance to use it as your kernel on your notebook. This however is tied to your compute instance and if you need to use the same setup on a different compute then you need to set it up again on a different instance. I have explained the setup on one of the previous threads, which can be helpful if you need one.

To summarize, if you are looking to create environments to train your models and infer then you will have to use the curated, user/system-managed environments in your experiments.
If you are looking to just use the notebooks then you can setup a custom kernel or virtual environment to run an experiment locally.

A great way to start learning about Azure ML through notebooks is to clone this repo on your ml.azure.com notebooks and follow the steps or tutorials to create and run experiments.

Once the repo is cloned from sample tab it will be available under files tab to be run on available compute and kernel.

If an answer is helpful, please click on or upvote which might help other community members reading this thread.