Unable to activate conda enviornment in AzureML when deploying pipeline with custom docker container

Question

I have trouble using the environment I've created in a custom docker container while deploying an AzureML pipeline.

Here is what I am doing:

1. I create a container which builds a conda enviornment along with other requirements.
2. The above container is create on Azure Pipelines and stored on a private azure container repository.
3. I pull this container in my pipeline using the below: aml_run_config.environment.docker.base_image = "xxxx"
   aml_run_config.environment.docker.base_image_registry.address = (
   "xxxx.azurecr.io"
   )
   aml_run_config.environment.docker.base_image_registry.username = (
   "xxxx"
   )
   aml_run_config.environment.docker.base_image_registry.password = (
   xxxx
   )

After that I tried two things:

1. Use a PythonScriptStep() to call my forecast.py
2. Use a CommandStep() with command = "conda run -n build_env python src/forecast.py"

build_env is the name of my conda env

In both cases I get a ModuleNotFoundError for pandas (the first library I use).

If I pull my docker container locally and try exactly the same command, it works fine.

What am I missing?

Answer 1

@David Clarance Thanks for the question. pre-built Azure ML environments: https://github.com/Azure/AzureML-Containers#base-image-dependencies

It is possible to use our own docker image (which is pushed into private container registry!)

Deploy models with custom Docker image - Azure Machine Learning | Microsoft Learn

To use an image from a private container registry that is not in your workspace, you must use docker.base_image_registry to specify the address of the repository and a user name and password:

  # Set the container registry information  
    myenv.docker.base_image_registry.address = "myregistry.azurecr.io"  
    myenv.docker.base_image_registry.username = "username"  
    myenv.docker.base_image_registry.password = "password"  
      
    myenv.inferencing_stack_version = "latest"  # This will install the inference specific apt packages.  
      
    # Define the packages needed by the model and scripts  
    from azureml.core.conda_dependencies import CondaDependencies  
    conda_dep = CondaDependencies()  
    # you must list azureml-defaults as a pip dependency  
    conda_dep.add_pip_package("azureml-defaults")  
    myenv.python.conda_dependencies=conda_dep