Authenticate clients for online endpoints
APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (current)
This article covers how to authenticate clients to perform control plane and data plane operations on online endpoints.
A control plane operation controls an endpoint and changes it. Control plane operations include create, read, update, and delete (CRUD) operations on online endpoints and online deployments.
A data plane operation uses data to interact with an online endpoint without changing the endpoint. For example, a data plane operation could consist of sending a scoring request to an online endpoint and getting a response.
Prerequisites
Before following the steps in this article, make sure you have the following prerequisites:
An Azure Machine Learning workspace. If you don't have one, use the steps in the Quickstart: Create workspace resources article to create one.
The Azure CLI and the
ml
extension or the Azure Machine Learning Python SDK v2:To install the Azure CLI and extension, see Install, set up, and use the CLI (v2).
Important
The CLI examples in this article assume that you are using the Bash (or compatible) shell. For example, from a Linux system or Windows Subsystem for Linux.
To install the Python SDK v2, use the following command:
pip install azure-ai-ml azure-identity
To update an existing installation of the SDK to the latest version, use the following command:
pip install --upgrade azure-ai-ml azure-identity
For more information, see Install the Python SDK v2 for Azure Machine Learning.
Limitations
Endpoints with Microsoft Entra token (aad_token
) auth mode don't support scoring using the CLI az ml online-endpoint invoke
, SDK ml_client.online_endpoints.invoke()
, or the Test or Consume tabs of the Azure Machine Learning studio. Instead, use a generic Python SDK or use REST API to pass the control plane token. For more information, see Score data using the key or token.
Prepare a user identity
You need a user identity to perform control plane operations (that is, CRUD operations) and data plane operations (that is, send scoring requests) on the online endpoint. You can use the same user identity or different user identities for the control plane and data plane operations. In this article, you use the same user identity for both control plane and data plane operations.
To create a user identity under Microsoft Entra ID, see Set up authentication. You'll need the identity ID later.
Assign permissions to the identity
In this section, you assign permissions to the user identity that you use for interacting with the endpoint. You begin by using either a built-in role or by creating a custom role. Thereafter, you assign the role to your user identity.
Use a built-in role
The AzureML Data Scientist
built-in role can be used to manage and use endpoints and deployments and it uses wildcards to include the following control plane RBAC actions:
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action
and to include the following data plane RBAC action:
Microsoft.MachineLearningServices/workspaces/onlineEndpoints/score/action
Optionally, the Azure Machine Learning Workspace Connection Secrets Reader
built-in role can be used to access secrets from workspace connections and it include the following control plane RBAC actions:
Microsoft.MachineLearningServices/workspaces/connections/listsecrets/action
Microsoft.MachineLearningServices/workspaces/metadata/secrets/read
If you use these built-in roles, there's no action needed at this step.
(Optional) Create a custom role
You can skip this step if you're using built-in roles or other pre-made custom roles.
Define the scope and actions for custom roles by creating JSON definitions of the roles. For example, the following role definition allows the user to CRUD an online endpoint, under a specified workspace.
custom-role-for-control-plane.json:
{ "Name": "Custom role for control plane operations - online endpoint", "IsCustom": true, "Description": "Can CRUD against online endpoints.", "Actions": [ "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action", "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action" ], "NotActions": [ ], "AssignableScopes": [ "/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>" ] }
The following role definition allows the user to send scoring requests to an online endpoint, under a specified workspace.
custom-role-for-scoring.json:
{ "Name": "Custom role for scoring - online endpoint", "IsCustom": true, "Description": "Can score against online endpoints.", "Actions": [ "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*/action" ], "NotActions": [ ], "AssignableScopes": [ "/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>" ] }
Use the JSON definitions to create custom roles:
az role definition create --role-definition custom-role-for-control-plane.json --subscription <subscriptionId> az role definition create --role-definition custom-role-for-scoring.json --subscription <subscriptionId>
Note
To create custom roles, you need one of three roles:
- owner
- user access administrator
- a custom role with
Microsoft.Authorization/roleDefinitions/write
permission (to create/update/delete custom roles) andMicrosoft.Authorization/roleDefinitions/read
permission (to view custom roles).
For more information on creating custom roles, see Azure custom roles.
Verify the role definition:
az role definition list --custom-role-only -o table az role definition list -n "Custom role for control plane operations - online endpoint" az role definition list -n "Custom role for scoring - online endpoint" export role_definition_id1=`(az role definition list -n "Custom role for control plane operations - online endpoint" --query "[0].id" | tr -d '"')` export role_definition_id2=`(az role definition list -n "Custom role for scoring - online endpoint" --query "[0].id" | tr -d '"')`
Assign the role to the identity
If you're using the
AzureML Data Scientist
built-in role, use the following code to assign the role to your user identity.az role assignment create --assignee <identityId> --role "AzureML Data Scientist" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
Optionally, if you're using the
Azure Machine Learning Workspace Connection Secrets Reader
built-in role, use the following code to assign the role to your user identity.az role assignment create --assignee <identityId> --role "Azure Machine Learning Workspace Connection Secrets Reader" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
If you're using a custom role, use the following code to assign the role to your user identity.
az role assignment create --assignee <identityId> --role "Custom role for control plane operations - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName> az role assignment create --assignee <identityId> --role "Custom role for scoring - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
Note
To assign custom roles to the user identity, you need one of three roles:
- owner
- user access administrator
- a custom role that allows
Microsoft.Authorization/roleAssignments/write
permission (to assign custom roles) andMicrosoft.Authorization/roleAssignments/read
(to view role assignments).
For more information on the different Azure roles and their permissions, see Azure roles and Assigning Azure roles using Azure Portal.
Confirm the role assignment:
az role assignment list --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>
Get the Microsoft Entra token for control plane operations
Perform this step if you plan to perform control plane operations with REST API, which will directly use the token.
If you plan to use other ways such as Azure Machine Learning CLI (v2), Python SDK (v2), or the Azure Machine Learning studio, you don't need to get the Microsoft Entra token manually. Rather, during sign in, your user identity would already be authenticated, and the token would automatically be retrieved and passed for you.
You can retrieve the Microsoft Entra token for control plane operations from the Azure resource endpoint: https://management.azure.com
.
Sign in to Azure.
az login
If you want to use a specific identity, use the following code to sign in with the identity:
az login --identity --username <identityId>
Use this context to get the token.
export CONTROL_PLANE_TOKEN=`(az account get-access-token --resource https://management.azure.com --query accessToken | tr -d '"')`
(Optional) Verify the resource endpoint and client ID for Microsoft Entra token
After you retrieve the Microsoft Entra token, you can verify that the token is for the right Azure resource endpoint management.azure.com
and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:
{
"aud": "https://management.azure.com",
"oid": "<your-object-id>"
}
Create an endpoint
The following example creates the endpoint with a system-assigned identity (SAI) as the endpoint identity. The SAI is the default identity type of the managed identity for endpoints. Some basic roles are automatically assigned for the SAI. For more information on role assignment for a system-assigned identity, see Automatic role assignment for endpoint identity.
The CLI doesn't require you to explicitly provide the control plane token. Instead, the CLI authenticates you during sign in, and the token is automatically retrieved and passed for you.
Create an endpoint definition YAML file.
endpoint.yml:
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json name: my-endpoint auth_mode: aad_token
You can replace
auth_mode
withkey
for key auth, oraml_token
for Azure Machine Learning token auth. In this example, you useaad_token
for Microsoft Entra token auth.az ml online-endpoint create -f endpoint.yml
Check the endpoint's status:
az ml online-endpoint show -n my-endpoint
If you want to override
auth_mode
(for example, toaad_token
) when creating an endpoint, run the following code:az ml online-endpoint create -n my-endpoint --auth_mode aad_token
If you want to update the existing endpoint and specify
auth_mode
(for example, toaad_token
), run the following code:az ml online-endpoint update -n my-endpoint --set auth_mode=aad_token
Create a deployment
To create a deployment, see Deploy an ML model with an online endpoint or Use REST to deploy an model as an online endpoint. There's no difference in how you create deployments for different auth modes.
The following code is an example of how to create a deployment. For more information on deploying online endpoints, see Deploy an ML model with an online endpoint (via CLI)
Create a deployment definition YAML file.
blue-deployment.yml:
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json name: blue endpoint_name: my-aad-auth-endp1 model: path: ../../model-1/model/ code_configuration: code: ../../model-1/onlinescoring/ scoring_script: score.py environment: conda_file: ../../model-1/environment/conda.yml image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest instance_type: Standard_DS3_v2 instance_count: 1
Create the deployment using the YAML file. For this example, set all traffic to the new deployment.
az ml online-deployment create -f blue-deployment.yml --all-traffic
Get the scoring URI for the endpoint
If you plan to use the CLI to invoke the endpoint, you're not required to get the scoring URI explicitly, as the CLI handles it for you. However, you can still use the CLI to get the scoring URI so that you can use it with other channels, such as REST API.
scoringUri=$(az ml online-endpoint show -n my-endpoint --query "scoring_uri")
Get the key or token for data plane operations
A key or token can be used for data plane operations, even though the process of getting the key or token is a control plane operation. In other words, you use a control plane token to get the key or token that you later use to perform your data plane operations.
Getting the key or Azure Machine Learning token requires that the correct role is assigned to the user identity that is requesting it, as described in authorization for control plane operations. The user identity doesn't need any extra roles to get the Microsoft Entra token.
Key or Azure Machine Learning token
If you plan to use the CLI to invoke the endpoint, and if the endpoint is set up to use an auth mode of key or Azure Machine Learning token (aml_token
), you're not required to get the data plane token explicitly, as the CLI handles it for you. However, you can still use the CLI to get the data plane token so that you can use it with other channels, such as REST API.
To get the key or Azure Machine Learning token (aml_token
), use the az ml online-endpoint get-credentials command. This command returns a JSON document that contains the key or Azure Machine Learning token.
Keys are returned in the primaryKey
and secondaryKey
fields. The following example shows how to use the --query
parameter to return only the primary key:
export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query primaryKey)
Azure Machine Learning Tokens are returned in the accessToken
field:
export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query accessToken)
Also, the expiryTimeUtc
and refreshAfterTimeUtc
fields contain the token expiration and refresh times.
Microsoft Entra token
To get the Microsoft Entra token (aad_token
) using CLI, use the az account get-access-token command. This command returns a JSON document that contains the Microsoft Entra token.
Microsoft Entra token is returned in the accessToken
field:
export DATA_PLANE_TOKEN=`(az account get-access-token --resource https://ml.azure.com --query accessToken | tr -d '"')`
Note
- The CLI
ml
extension doesn't support getting the Microsoft Entra token. Useaz account get-access-token
instead, as described in the previous code. - The token for data plane operations is retrieved from the Azure resource endpoint
ml.azure.com
instead ofmanagement.azure.com
, unlike the token for control plane operations.
Verify the resource endpoint and client ID for the Microsoft Entra token
After getting the Entra token, you can verify that the token is for the right Azure resource endpoint ml.azure.com
and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:
{
"aud": "https://ml.azure.com",
"oid": "<your-object-id>"
}
Score data using the key or token
Key or Azure Machine Learning token
You can use az ml online-endpoint invoke
for endpoints with a key or Azure Machine Learning token. The CLI handles the key or Azure Machine Learning token automatically so you don't need to pass it explicitly.
az ml online-endpoint invoke -n my-endpoint -r request.json
Microsoft Entra token
Using az ml online-endpoint invoke
for endpoints with a Microsoft Entra token isn't supported. Use REST API instead, and use the endpoint's scoring URI to invoke the endpoint.
Log and monitor traffic
To enable traffic logging in the diagnostics settings for the endpoint, follow the steps in How to enable/disable logs.
If the diagnostic setting is enabled, you can check the AmlOnlineEndpointTrafficLogs
table to see the auth mode and user identity.
Related content
Feedback
https://aka.ms/ContentUserFeedback.
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see:Submit and view feedback for