Events
Take the Microsoft Learn Challenge
Nov 19, 11 PM - Jan 10, 11 PM
Ignite Edition - Build skills in Microsoft Azure and earn a digital badge by January 10!
Register nowThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (current)
This article covers how to authenticate clients to perform control plane and data plane operations on online endpoints.
A control plane operation controls an endpoint and changes it. Control plane operations include create, read, update, and delete (CRUD) operations on online endpoints and online deployments.
A data plane operation uses data to interact with an online endpoint without changing the endpoint. For example, a data plane operation could consist of sending a scoring request to an online endpoint and getting a response.
Before following the steps in this article, make sure you have the following prerequisites:
An Azure Machine Learning workspace. If you don't have one, use the steps in the Quickstart: Create workspace resources article to create one.
The Azure CLI and the ml
extension or the Azure Machine Learning Python SDK v2:
To install the Azure CLI and extension, see Install, set up, and use the CLI (v2).
Important
The CLI examples in this article assume that you are using the Bash (or compatible) shell. For example, from a Linux system or Windows Subsystem for Linux.
To install the Python SDK v2, use the following command:
pip install azure-ai-ml azure-identity
To update an existing installation of the SDK to the latest version, use the following command:
pip install --upgrade azure-ai-ml azure-identity
For more information, see Install the Python SDK v2 for Azure Machine Learning.
You need a user identity to perform control plane operations (that is, CRUD operations) and data plane operations (that is, send scoring requests) on the online endpoint. You can use the same user identity or different user identities for the control plane and data plane operations. In this article, you use the same user identity for both control plane and data plane operations.
To create a user identity under Microsoft Entra ID, see Set up authentication. You'll need the identity ID later.
In this section, you assign permissions to the user identity that you use for interacting with the endpoint. You begin by using either a built-in role or by creating a custom role. Thereafter, you assign the role to your user identity.
The AzureML Data Scientist
built-in role can be used to manage and use endpoints and deployments and it uses wildcards to include the following control plane RBAC actions:
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action
and to include the following data plane RBAC action:
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/score/action
Optionally, the Azure Machine Learning Workspace Connection Secrets Reader
built-in role can be used to access secrets from workspace connections and it include the following control plane RBAC actions:
Microsoft.MachineLearningServices/workspaces/connections/listsecrets/action
Microsoft.MachineLearningServices/workspaces/metadata/secrets/read
If you use these built-in roles, there's no action needed at this step.
You can skip this step if you're using built-in roles or other pre-made custom roles.
Define the scope and actions for custom roles by creating JSON definitions of the roles. For example, the following role definition allows the user to CRUD an online endpoint, under a specified workspace.
custom-role-for-control-plane.json:
{
"Name": "Custom role for control plane operations - online endpoint",
"IsCustom": true,
"Description": "Can CRUD against online endpoints.",
"Actions": [
"Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write",
"Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete",
"Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read",
"Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action",
"Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action",
"Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action"
],
"NotActions": [
],
"AssignableScopes": [
"/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>"
]
}
The following role definition allows the user to send scoring requests to an online endpoint, under a specified workspace.
custom-role-for-scoring.json:
{
"Name": "Custom role for scoring - online endpoint",
"IsCustom": true,
"Description": "Can score against online endpoints.",
"Actions": [
"Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*/action"
],
"NotActions": [
],
"AssignableScopes": [
"/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>"
]
}
Use the JSON definitions to create custom roles:
az role definition create --role-definition custom-role-for-control-plane.json --subscription <subscriptionId>
az role definition create --role-definition custom-role-for-scoring.json --subscription <subscriptionId>
Note
To create custom roles, you need one of three roles:
Microsoft.Authorization/roleDefinitions/write
permission (to create/update/delete custom roles) and Microsoft.Authorization/roleDefinitions/read
permission (to view custom roles).For more information on creating custom roles, see Azure custom roles.
Verify the role definition:
az role definition list --custom-role-only -o table
az role definition list -n "Custom role for control plane operations - online endpoint"
az role definition list -n "Custom role for scoring - online endpoint"
export role_definition_id1=`(az role definition list -n "Custom role for control plane operations - online endpoint" --query "[0].id" | tr -d '"')`
export role_definition_id2=`(az role definition list -n "Custom role for scoring - online endpoint" --query "[0].id" | tr -d '"')`
If you're using the AzureML Data Scientist
built-in role, use the following code to assign the role to your user identity.
az role assignment create --assignee <identityId> --role "AzureML Data Scientist" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
Optionally, if you're using the Azure Machine Learning Workspace Connection Secrets Reader
built-in role, use the following code to assign the role to your user identity.
az role assignment create --assignee <identityId> --role "Azure Machine Learning Workspace Connection Secrets Reader" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
If you're using a custom role, use the following code to assign the role to your user identity.
az role assignment create --assignee <identityId> --role "Custom role for control plane operations - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
az role assignment create --assignee <identityId> --role "Custom role for scoring - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
Note
To assign custom roles to the user identity, you need one of three roles:
Microsoft.Authorization/roleAssignments/write
permission (to assign custom roles) and Microsoft.Authorization/roleAssignments/read
(to view role assignments).For more information on the different Azure roles and their permissions, see Azure roles and Assigning Azure roles using Azure Portal.
Confirm the role assignment:
az role assignment list --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
Perform this step if you plan to perform control plane operations with REST API, which will directly use the token.
If you plan to use other ways such as Azure Machine Learning CLI (v2), Python SDK (v2), or the Azure Machine Learning studio, you don't need to get the Microsoft Entra token manually. Rather, during sign in, your user identity would already be authenticated, and the token would automatically be retrieved and passed for you.
You can retrieve the Microsoft Entra token for control plane operations from the Azure resource endpoint: https://management.azure.com
.
Sign in to Azure.
az login
If you want to use a specific identity, use the following code to sign in with the identity:
az login --identity --username <identityId>
Use this context to get the token.
export CONTROL_PLANE_TOKEN=`(az account get-access-token --resource https://management.azure.com --query accessToken | tr -d '"')`
After you retrieve the Microsoft Entra token, you can verify that the token is for the right Azure resource endpoint management.azure.com
and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:
{
"aud": "https://management.azure.com",
"oid": "<your-object-id>"
}
The following example creates the endpoint with a system-assigned identity (SAI) as the endpoint identity. The SAI is the default identity type of the managed identity for endpoints. Some basic roles are automatically assigned for the SAI. For more information on role assignment for a system-assigned identity, see Automatic role assignment for endpoint identity.
The CLI doesn't require you to explicitly provide the control plane token. Instead, the CLI az login
authenticates you during sign in, and the token is automatically retrieved and passed for you.
Create an endpoint definition YAML file.
endpoint.yml:
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: my-endpoint
auth_mode: aad_token
You can replace auth_mode
with key
for key auth, or aml_token
for Azure Machine Learning token auth. In this example, you use aad_token
for Microsoft Entra token auth.
az ml online-endpoint create -f endpoint.yml
Check the endpoint's status:
az ml online-endpoint show -n my-endpoint
If you want to override auth_mode
(for example, to aad_token
) when creating an endpoint, run the following code:
az ml online-endpoint create -n my-endpoint --auth_mode aad_token
If you want to update the existing endpoint and specify auth_mode
(for example, to aad_token
), run the following code:
az ml online-endpoint update -n my-endpoint --set auth_mode=aad_token
To create a deployment, see Deploy an ML model with an online endpoint or Use REST to deploy an model as an online endpoint. There's no difference in how you create deployments for different auth modes.
The following code is an example of how to create a deployment. For more information on deploying online endpoints, see Deploy an ML model with an online endpoint (via CLI)
Create a deployment definition YAML file.
blue-deployment.yml:
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: my-aad-auth-endp1
model:
path: ../../model-1/model/
code_configuration:
code: ../../model-1/onlinescoring/
scoring_script: score.py
environment:
conda_file: ../../model-1/environment/conda.yml
image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest
instance_type: Standard_DS3_v2
instance_count: 1
Create the deployment using the YAML file. For this example, set all traffic to the new deployment.
az ml online-deployment create -f blue-deployment.yml --all-traffic
If you plan to use the CLI to invoke the endpoint, you're not required to get the scoring URI explicitly, as the CLI handles it for you. However, you can still use the CLI to get the scoring URI so that you can use it with other channels, such as REST API.
scoringUri=$(az ml online-endpoint show -n my-endpoint --query "scoring_uri")
A key or token can be used for data plane operations, even though the process of getting the key or token is a control plane operation. In other words, you use a control plane token to get the key or token that you later use to perform your data plane operations.
Getting the key or Azure Machine Learning token requires that the correct role is assigned to the user identity that is requesting it, as described in authorization for control plane operations. Getting the Microsoft Entra token doesn't require any extra roles for the user identity.
If you plan to use the CLI to invoke the endpoint, you're not required to get the keys or token for data plane operations explicitly, as the CLI handles it for you. However, you can still use the CLI to get the keys or token for data plane operation so that you can use it with other channels, such as REST API.
To get the keys or token for data plane operations, use the az ml online-endpoint get-credentials command. This command returns a JSON output that contains the keys, token, and/or additional information.
Tip
To extract a specific information from the JSON output, the --query
parameter of the CLI command is used as an example. However, you can use any suitable tool for this purpose.
When auth_mode
of the endpoint is key
primaryKey
and secondaryKey
fields.export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query primaryKey)
export DATA_PLANE_TOKEN2=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query secondaryKey)
When auth_mode
of the endpoint is aml_token
accessToken
field.expiryTimeUtc
field.refreshAfterTimeUtc
field.export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query accessToken)
export EXPIRY_TIME_UTC=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query expiryTimeUtc)
export REFRESH_AFTER_TIME_UTC=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query refreshAfterTimeUtc)
When auth_mode
of the endpoint is aad_token
accessToken
field.expiryTimeUtc
field.export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query accessToken)
export EXPIRY_TIME_UTC=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query expiryTimeUtc)
After getting the Entra token, you can verify that the token is for the right Azure resource endpoint ml.azure.com
and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:
{
"aud": "https://ml.azure.com",
"oid": "<your-object-id>"
}
You can use az ml online-endpoint invoke
for endpoints with a key, an Azure Machine Learning token, or a Microsoft Entra token. The CLI handles the key or token automatically so you don't need to pass it explicitly.
az ml online-endpoint invoke -n my-endpoint -r request.json
To enable traffic logging in the diagnostics settings for the endpoint, follow the steps in How to enable/disable logs.
If the diagnostic setting is enabled, you can check the AmlOnlineEndpointTrafficLogs
table to see the auth mode and user identity.
Events
Take the Microsoft Learn Challenge
Nov 19, 11 PM - Jan 10, 11 PM
Ignite Edition - Build skills in Microsoft Azure and earn a digital badge by January 10!
Register nowTraining
Module
Control authentication for your APIs with Azure API Management - Training
Discover how to protect your APIs from unauthorized use with API keys and client certificate authentication.
Certification
Microsoft 365 Certified: Endpoint Administrator Associate - Certifications
Plan and execute an endpoint deployment strategy, using essential elements of modern management, co-management approaches, and Microsoft Intune integration.