ML Studio: Environment and Dependency Management Challenges

Question

Hi team,

This is another ticket connected to our usage of Azure ML Studio workspaces

Other in this series:

• https://learn.microsoft.com/en-us/answers/questions/2285569/

So, this ticket refers to another feedback case by our engineers:

• Issue Description:
  • Difficulty accessing private pip repositories via Personal Access Tokens (PAT) when building environments using custom Dockerfiles.
  Lack of a documented and secure way to leverage secrets stored in Azure ML connections within Docker build processes.
• Suggested Solutions:
  • Document and enable secure access to Azure ML-stored secrets during custom Docker environment builds.
  • Improve integration between Azure DevOps Artifacts and Azure ML environment-building workflows.

As usual, thank you for your feedback and help.

Arturo

Answer 1

Hi Arturo Sánchez Pineda,

Thank you for raising this issue and providing detailed context.

We understand that you're facing difficulty securely accessing private pip repositories (e.g., Azure DevOps Artifacts) via PAT during custom Docker builds in Azure ML Studio.

Since Docker builds happen in isolated environments, Azure ML connections/secrets (including Key Vault and managed identities) are not available by default at build time. To resolve this:

Recommended Approach:

·   Store your PAT token in Azure Key Vault.

·   Use Azure DevOps Pipelines to inject this secret into the Docker build as a build argument (--build-arg).

·   Modify your Dockerfile to read this argument and install private packages accordingly.

Alternate Options:

·   Use multi-stage Docker builds and inject secrets later via environment variables during runtime in Azure ML.

·   Explore using Managed Identity if supported by Azure DevOps Artifacts.

We also acknowledge your feedback on improving documentation and tighter integration between Azure ML and Azure DevOps for such workflows and will share this feedback to the product team.

Please refer below documents for better understanding:

Secret injection in online endpoints (preview)

Building custom Docker images for training and deployment

Leverage private packages when training and deploying models

Hope this helps.

Thank you!