Azure Machine Learning Studio, Python 3.10

Question

Hi,

I am trying to train my Deep Learning model in Azure ML studio through the Visual Studio Code extension. My code is build upon Python 3.10, which is needed for several packages. However I cannot seem to get my Azure ML Studio compute to run on this version of Python, as it remains in its default of 3.8.5.

Setting an environment does not work as well, at least the way I tried it. As this process is not well documented elsewhere and nowhere on the internet I can find a solution to my problem.

Thank you in advance for your help!

Best regards,
Remco Coppens

Answer 1

@Remco Coppens I believe you use the compute instance while using the VS code extension. If this is the case, did you try to enable SSH to your compute instance to login and try to upgrade the python version or set the required version to use by default?

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 2

Have you tried creating a custom environment? Maybe you need to specify explicitly Python 3.10 and configure your training job to use this environment.

You can define an environment in Azure ML using the Python SDK.

https://learn.microsoft.com/en-us/azure/machine-learning/concept-environments?view=azureml-api-2

from azureml.core import Environment
from azureml.core.conda_dependencies import CondaDependencies
# Create a Python environment for AzureML
env = Environment('custom-python3.10-env')
# Build conda dependencies
conda_dep = CondaDependencies()
conda_dep.set_python_version('3.10')
conda_dep.add_conda_package('numpy')  # Add other necessary packages
conda_dep.add_pip_package('tensorflow')  # Assuming TensorFlow is required
conda_dep.add_pip_package('keras')  # Add any additional pip packages
# Add the dependencies to the environment
env.python.conda_dependencies = conda_dep
# Register the environment to reuse later env.register(workspace=your_workspace)

How to use it ?

from azureml.core import Experiment, ScriptRunConfig

# Set up the script run configuration
src = ScriptRunConfig(source_directory='./your_script_folder',
                      script='train.py',  # Your training script
                      compute_target='your-compute-target',  # Your compute target
                      environment=env)  # Use the custom environment

# Submit the experiment
experiment = Experiment(workspace=your_workspace, name='MyExperiment')
run = experiment.submit(config=src)
run.wait_for_completion(show_output=True)