DavidClarance-7033 avatar image
0 Votes"
DavidClarance-7033 asked ramr-msft commented

Unable to activate conda enviornment in AzureML when deploying pipeline with custom docker container

I have trouble using the environment I've created in a custom docker container while deploying an AzureML pipeline.

Here is what I am doing:

  1. I create a container which builds a conda enviornment along with other requirements.

  2. The above container is create on Azure Pipelines and stored on a private azure container repository.

  3. I pull this container in my pipeline using the below:

aml_run_config.environment.docker.base_image = "xxxx"
    aml_run_config.environment.docker.base_image_registry.address = (
    aml_run_config.environment.docker.base_image_registry.username = (
    aml_run_config.environment.docker.base_image_registry.password = (

After that I tried two things:

  1. Use a PythonScriptStep() to call my

  2. Use a CommandStep() with command = "conda run -n build_env python src/"

build_env is the name of my conda env

In both cases I get a ModuleNotFoundError for pandas (the first library I use).

If I pull my docker container locally and try exactly the same command, it works fine.

What am I missing?

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@DavidClarance-7033 Thanks, Could you please share the error logs.

0 Votes 0 ·

1 Answer

ramr-msft avatar image
0 Votes"
ramr-msft answered ramr-msft commented

@DavidClarance-7033 Thanks for the question. pre-built Azure ML environments:

It is possible to use our own docker image (which is pushed into private container registry!)

Deploy models with custom Docker image - Azure Machine Learning | Microsoft Docs

To use an image from a private container registry that is not in your workspace, you must use docker.base_image_registry to specify the address of the repository and a user name and password:

   # Set the container registry information
     myenv.docker.base_image_registry.address = ""
     myenv.docker.base_image_registry.username = "username"
     myenv.docker.base_image_registry.password = "password"
     myenv.inferencing_stack_version = "latest"  # This will install the inference specific apt packages.
     # Define the packages needed by the model and scripts
     from azureml.core.conda_dependencies import CondaDependencies
     conda_dep = CondaDependencies()
     # you must list azureml-defaults as a pip dependency

· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks @ramr-msft. I tried add the statements as you suggest but I still get the same error. I realize I'm adding packages at different stages so maybe that's the issue.

  1. I add packages in the docker container.

  2. I add a private package in the azure pipeline that builds the docker container using the Docker@2 task.

  3. I add azureml-defaults (as per your recommendation) in the CondaDependencies() workflow.

Here's the error message:

     "error": {
         "code": "UserError",
         "message": "User program failed with ModuleNotFoundError: No module named 'pandas'",
         "messageParameters": {},
         "detailsUri": "",
         "details": []
     "time": "0001-01-01T00:00:00.000Z"
         InnerException None
     "error": {
         "message": "Activity Failed:\n{\n    \"error\": {\n        \"code\": \"UserError\",\n        \"message\": \"User program failed with ModuleNotFoundError: No module named 'pandas'\",\n
     \"messageParameters\": {},\n        \"detailsUri\": \"\",\n        \"details\": []\n    },\n    \"time\": \"0001-01-01T00:00:00.000Z\"\n}"

0 Votes 0 ·

@DavidClarance-7033 Thanks for the details. Here is the troubleshooting doc for module not found error that could help to debug.

0 Votes 0 ·