Authenticate clients for online endpoints
APPLIES TO:
Azure CLI ml extension v2 (current)
Python SDK azure-ai-ml v2 (current)
This article covers how to authenticate clients to perform control plane and data plane operations on online endpoints.
A control plane operation controls an endpoint and changes it. Control plane operations include create, read, update, and delete (CRUD) operations on online endpoints and online deployments.
A data plane operation uses data to interact with an online endpoint without changing the endpoint. For example, a data plane operation could consist of sending a scoring request to an online endpoint and getting a response.
Prerequisites
Before following the steps in this article, make sure you have the following prerequisites:
An Azure Machine Learning workspace. If you don't have one, use the steps in the Quickstart: Create workspace resources article to create one.
The Azure CLI and the
ml
extension or the Azure Machine Learning Python SDK v2:To install the Azure CLI and extension, see Install, set up, and use the CLI (v2).
Important
The CLI examples in this article assume that you are using the Bash (or compatible) shell. For example, from a Linux system or Windows Subsystem for Linux.
To install the Python SDK v2, use the following command:
pip install azure-ai-ml azure-identity
To update an existing installation of the SDK to the latest version, use the following command:
pip install --upgrade azure-ai-ml azure-identity
For more information, see Install the Python SDK v2 for Azure Machine Learning.
Prepare a user identity
You need a user identity to perform control plane operations (that is, CRUD operations) and data plane operations (that is, send scoring requests) on the online endpoint. You can use the same user identity or different user identities for the control plane and data plane operations. In this article, you use the same user identity for both control plane and data plane operations.
To create a user identity under Microsoft Entra ID, see Set up authentication. You'll need the identity ID later.
Assign permissions to the identity
In this section, you assign permissions to the user identity that you use for interacting with the endpoint. You begin by using either a built-in role or by creating a custom role. Thereafter, you assign the role to your user identity.
Use a built-in role
The AzureML Data Scientist
built-in role can be used to manage and use endpoints and deployments and it uses wildcards to include the following control plane RBAC actions:
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action
and to include the following data plane RBAC action:
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/score/action
Optionally, the Azure Machine Learning Workspace Connection Secrets Reader
built-in role can be used to access secrets from workspace connections and it include the following control plane RBAC actions:
Microsoft.MachineLearningServices/workspaces/connections/listsecrets/action
Microsoft.MachineLearningServices/workspaces/metadata/secrets/read
If you use these built-in roles, there's no action needed at this step.
(Optional) Create a custom role
You can skip this step if you're using built-in roles or other pre-made custom roles.
Define the scope and actions for custom roles by creating JSON definitions of the roles. For example, the following role definition allows the user to CRUD an online endpoint, under a specified workspace.
custom-role-for-control-plane.json:
{ "Name": "Custom role for control plane operations - online endpoint", "IsCustom": true, "Description": "Can CRUD against online endpoints.", "Actions": [ "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action" ], "NotActions": [ ], "AssignableScopes": [ "/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>" ] }
The following role definition allows the user to send scoring requests to an online endpoint, under a specified workspace.
custom-role-for-scoring.json:
{ "Name": "Custom role for scoring - online endpoint", "IsCustom": true, "Description": "Can score against online endpoints.", "Actions": [ "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*/action" ], "NotActions": [ ], "AssignableScopes": [ "/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>" ] }
Use the JSON definitions to create custom roles:
az role definition create --role-definition custom-role-for-control-plane.json --subscription <subscriptionId> az role definition create --role-definition custom-role-for-scoring.json --subscription <subscriptionId>
Note
To create custom roles, you need one of three roles:
- owner
- user access administrator
- a custom role with
Microsoft.Authorization/roleDefinitions/write
permission (to create/update/delete custom roles) andMicrosoft.Authorization/roleDefinitions/read
permission (to view custom roles).
For more information on creating custom roles, see Azure custom roles.
Verify the role definition:
az role definition list --custom-role-only -o table az role definition list -n "Custom role for control plane operations - online endpoint" az role definition list -n "Custom role for scoring - online endpoint" export role_definition_id1=`(az role definition list -n "Custom role for control plane operations - online endpoint" --query "[0].id" | tr -d '"')` export role_definition_id2=`(az role definition list -n "Custom role for scoring - online endpoint" --query "[0].id" | tr -d '"')`
Assign the role to the identity
If you're using the
AzureML Data Scientist
built-in role, use the following code to assign the role to your user identity.az role assignment create --assignee <identityId> --role "AzureML Data Scientist" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
Optionally, if you're using the
Azure Machine Learning Workspace Connection Secrets Reader
built-in role, use the following code to assign the role to your user identity.az role assignment create --assignee <identityId> --role "Azure Machine Learning Workspace Connection Secrets Reader" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
If you're using a custom role, use the following code to assign the role to your user identity.
az role assignment create --assignee <identityId> --role "Custom role for control plane operations - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName> az role assignment create --assignee <identityId> --role "Custom role for scoring - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
Note
To assign custom roles to the user identity, you need one of three roles:
- owner
- user access administrator
- a custom role that allows
Microsoft.Authorization/roleAssignments/write
permission (to assign custom roles) andMicrosoft.Authorization/roleAssignments/read
(to view role assignments).
For more information on the different Azure roles and their permissions, see Azure roles and Assigning Azure roles using Azure Portal.
Confirm the role assignment:
az role assignment list --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
Get the Microsoft Entra token for control plane operations
Perform this step if you plan to perform control plane operations with REST API, which will directly use the token.
If you plan to use other ways such as Azure Machine Learning CLI (v2), Python SDK (v2), or the Azure Machine Learning studio, you don't need to get the Microsoft Entra token manually. Rather, during sign in, your user identity would already be authenticated, and the token would automatically be retrieved and passed for you.
You can retrieve the Microsoft Entra token for control plane operations from the Azure resource endpoint: https://management.azure.com
.
Sign in to Azure.
az login
If you want to use a specific identity, use the following code to sign in with the identity:
az login --identity --username <identityId>
Use this context to get the token.
export CONTROL_PLANE_TOKEN=`(az account get-access-token --resource https://management.azure.com --query accessToken | tr -d '"')`
(Optional) Verify the resource endpoint and client ID for Microsoft Entra token
After you retrieve the Microsoft Entra token, you can verify that the token is for the right Azure resource endpoint management.azure.com
and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:
{
"aud": "https://management.azure.com",
"oid": "<your-object-id>"
}
Create an endpoint
The following example creates the endpoint with a system-assigned identity (SAI) as the endpoint identity. The SAI is the default identity type of the managed identity for endpoints. Some basic roles are automatically assigned for the SAI. For more information on role assignment for a system-assigned identity, see Automatic role assignment for endpoint identity.
The CLI doesn't require you to explicitly provide the control plane token. Instead, the CLI az login
authenticates you during sign in, and the token is automatically retrieved and passed for you.
Create an endpoint definition YAML file.
endpoint.yml:
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json name: my-endpoint auth_mode: aad_token
You can replace
auth_mode
withkey
for key auth, oraml_token
for Azure Machine Learning token auth. In this example, you useaad_token
for Microsoft Entra token auth.az ml online-endpoint create -f endpoint.yml
Check the endpoint's status:
az ml online-endpoint show -n my-endpoint
If you want to override
auth_mode
(for example, toaad_token
) when creating an endpoint, run the following code:az ml online-endpoint create -n my-endpoint --auth_mode aad_token
If you want to update the existing endpoint and specify
auth_mode
(for example, toaad_token
), run the following code:az ml online-endpoint update -n my-endpoint --set auth_mode=aad_token
Create a deployment
To create a deployment, see Deploy an ML model with an online endpoint or Use REST to deploy an model as an online endpoint. There's no difference in how you create deployments for different auth modes.
The following code is an example of how to create a deployment. For more information on deploying online endpoints, see Deploy an ML model with an online endpoint (via CLI)
Create a deployment definition YAML file.
blue-deployment.yml:
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json name: blue endpoint_name: my-aad-auth-endp1 model: path: ../../model-1/model/ code_configuration: code: ../../model-1/onlinescoring/ scoring_script: score.py environment: conda_file: ../../model-1/environment/conda.yml image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest instance_type: Standard_DS3_v2 instance_count: 1
Create the deployment using the YAML file. For this example, set all traffic to the new deployment.
az ml online-deployment create -f blue-deployment.yml --all-traffic
Get the scoring URI for the endpoint
If you plan to use the CLI to invoke the endpoint, you're not required to get the scoring URI explicitly, as the CLI handles it for you. However, you can still use the CLI to get the scoring URI so that you can use it with other channels, such as REST API.
scoringUri=$(az ml online-endpoint show -n my-endpoint --query "scoring_uri")
Get the key or token for data plane operations
A key or token can be used for data plane operations, even though the process of getting the key or token is a control plane operation. In other words, you use a control plane token to get the key or token that you later use to perform your data plane operations.
Getting the key or Azure Machine Learning token requires that the correct role is assigned to the user identity that is requesting it, as described in authorization for control plane operations. Getting the Microsoft Entra token doesn't require any extra roles for the user identity.
If you plan to use the CLI to invoke the endpoint, you're not required to get the keys or token for data plane operations explicitly, as the CLI handles it for you. However, you can still use the CLI to get the keys or token for data plane operation so that you can use it with other channels, such as REST API.
To get the keys or token for data plane operations, use the az ml online-endpoint get-credentials command. This command returns a JSON output that contains the keys, token, and/or additional information.
Tip
To extract a specific information from the JSON output, the --query
parameter of the CLI command is used as an example. However, you can use any suitable tool for this purpose.
When auth_mode
of the endpoint is key
- Keys are returned in the
primaryKey
andsecondaryKey
fields.
export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query primaryKey)
export DATA_PLANE_TOKEN2=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query secondaryKey)
When auth_mode
of the endpoint is aml_token
- Token is returned in the
accessToken
field. - Token expiration time is returned in the
expiryTimeUtc
field. - Token refresh time is returned in the
refreshAfterTimeUtc
field.
export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query accessToken)
export EXPIRY_TIME_UTC=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query expiryTimeUtc)
export REFRESH_AFTER_TIME_UTC=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query refreshAfterTimeUtc)
When auth_mode
of the endpoint is aad_token
- Token is returned in the
accessToken
field. - Token expiration time is returned in the
expiryTimeUtc
field.
export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query accessToken)
export EXPIRY_TIME_UTC=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query expiryTimeUtc)
Verify the resource endpoint and client ID for the Microsoft Entra token
After getting the Entra token, you can verify that the token is for the right Azure resource endpoint ml.azure.com
and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:
{
"aud": "https://ml.azure.com",
"oid": "<your-object-id>"
}
Score data using the key or token
You can use az ml online-endpoint invoke
for endpoints with a key, an Azure Machine Learning token, or a Microsoft Entra token. The CLI handles the key or token automatically so you don't need to pass it explicitly.
az ml online-endpoint invoke -n my-endpoint -r request.json
Log and monitor traffic
To enable traffic logging in the diagnostics settings for the endpoint, follow the steps in How to enable/disable logs.
If the diagnostic setting is enabled, you can check the AmlOnlineEndpointTrafficLogs
table to see the auth mode and user identity.