Edit

Share via

Secret injection in online endpoints (preview)

APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (current)

Secret injection in online endpoints helps you securely use API keys and other secrets in your deployments without exposing them in your code. This article explains:

  • Why secret injection matters for secure deployments
  • How to use managed identities to retrieve secrets
  • How the secret injection feature simplifies the process

By the end, you'll understand which approach works best for your scenario.

Secret injection in the context of an online endpoint is a process of retrieving secrets, such as API keys, from secret stores and injecting them into your user container that runs inside an online deployment. The inference server that runs your scoring script or the inferencing stack that you bring with a BYOC (bring your own container) deployment approach securely accesses secrets through environment variables.

Important

This feature is currently in public preview. This preview version is provided without a service-level agreement, and we don't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.

For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

Problem statement

When you create an online deployment, you might want to use secrets from within the deployment to access external services. Some of these external services include Microsoft Azure OpenAI service, Foundry Tools, and Azure AI Content Safety.

To use the secrets, you need a way to securely pass them to your user container that runs inside the deployment. Don't include secrets as part of the deployment definition, since this practice exposes the secrets in the deployment definition.

A better approach is to store the secrets in secret stores and then retrieve them securely from within the deployment. However, this approach poses its own challenge: how the deployment should authenticate itself to the secret stores to retrieve secrets. Because the online deployment runs your user container by using the endpoint identity, which is a managed identity, you can use Azure RBAC to control the endpoint identity's permissions and allow the endpoint to retrieve secrets from the secret stores. Using this approach requires you to complete the following tasks:

  • Assign the right roles to the endpoint identity so that it can read secrets from the secret stores.
  • Implement the scoring logic for the deployment so that it uses the endpoint's managed identity to retrieve the secrets from the secret stores.

While this approach of using a managed identity is a secure way to retrieve and inject secrets, secret injection via the secret injection feature further simplifies the process of retrieving secrets for workspace connections and key vaults.

Managed identity associated with the endpoint

An online deployment runs your user container with the managed identity associated with the endpoint. This managed identity, called the endpoint identity, is a Microsoft Entra ID that supports Azure RBAC. Therefore, you can assign Azure roles to the identity to control permissions that are required to perform operations. The endpoint identity can be either a system-assigned identity (SAI) or a user-assigned identity (UAI). You decide which of these kinds of identities to use when you create the endpoint.

  • For a system-assigned identity, the identity is created automatically when you create the endpoint, and roles with fundamental permissions (such as the Azure Container Registry pull permission and the storage blob data reader) are automatically assigned.
  • For a user-assigned identity, you need to create the identity first, and then associate it with the endpoint when you create the endpoint. You're also responsible for assigning proper roles to the UAI as needed.

For more information on using managed identities of an endpoint, see How to access resources from endpoints with managed identities, and the example for using managed identities to interact with external services.

Role assignment to the endpoint identity

The following roles are required by the secret stores:

  • For secrets stored in workspace connections under your workspace: Workspace Connections provides a List Secrets API (preview) that requires the identity that calls the API to have Azure Machine Learning Workspace Connection Secrets Reader role (or equivalent) assigned to the identity.
  • For secrets stored in an external Microsoft Azure Key Vault: Key Vault provides a Get Secret Versions API that requires the identity that calls the API to have Key Vault Secrets User role (or equivalent) assigned to the identity.

Implementation of secret injection

After you retrieve secrets, such as API keys, from secret stores, you can inject them into a user container that runs inside the online deployment in two ways:

Choose the approach that best fits your security requirements and development workflow.

Both approaches involve two steps:

  1. Retrieve secrets from the secret stores, by using the endpoint identity.
  2. Inject the secrets into your user container.

Secret injection by using managed identities

In your deployment definition, use the endpoint identity to call the APIs from secret stores. You can implement this logic either in your scoring script or in shell scripts that you run in your BYOC container. For more information about implementing secret injection by using managed identities, see the example for using managed identities to interact with external services.

Secret injection via the secret injection feature

To use the secret injection feature, map the secrets you want to refer to from workspace connections or the Key Vault onto the environment variables in your deployment definition. This approach doesn't require you to write any code in your scoring script or in shell scripts that you run in your BYOC container. To map the secrets from workspace connections or the Key Vault onto the environment variables, the following conditions must be met:

  • During endpoint creation, if you define an online endpoint to enforce access to default secret stores (workspace connections under the current workspace), your user identity that creates the deployment under the endpoint must have the permissions to read secrets from workspace connections.
  • The endpoint identity that the deployment uses must have permissions to read secrets from either workspace connections or the Key Vault, as referenced in the deployment definition.

Note

  • If you successfully create the endpoint with an SAI and set the flag to enforce access to default secret stores, the endpoint automatically has permission for workspace connections.
  • If the endpoint uses a UAI, or you don't set the flag to enforce access to default secret stores, the endpoint identity might not have permission for workspace connections. In this situation, you need to manually assign the role for the workspace connections to the endpoint identity.
  • The endpoint identity doesn't automatically receive permission for the external Key Vault. If you're using the Key Vault as a secret store, you need to manually assign the role for the Key Vault to the endpoint identity.

For more information on using secret injection, see Deploy machine learning models to online endpoints with secret injection (preview).

Next steps

Now that you understand secret injection concepts, learn how to implement it:

Ready to get started? Follow the deployment guide.

  • Last updated on