Edit

Authenticate clients for online endpoints

  • Article

APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (current)

This article covers how to authenticate clients to perform control plane and data plane operations on online endpoints.

A control plane operation controls an endpoint and changes it. Control plane operations include create, read, update, and delete (CRUD) operations on online endpoints and online deployments.

A data plane operation uses data to interact with an online endpoint without changing the endpoint. For example, a data plane operation could consist of sending a scoring request to an online endpoint and getting a response.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

  • An Azure Machine Learning workspace. If you don't have one, use the steps in the Quickstart: Create workspace resources article to create one.

  • The Azure CLI and the ml extension or the Azure Machine Learning Python SDK v2:

Limitations

Endpoints with Microsoft Entra token (aad_token) auth mode don't support scoring using the CLI az ml online-endpoint invoke, SDK ml_client.online_endpoints.invoke(), or the Test or Consume tabs of the Azure Machine Learning studio. Instead, use a generic Python SDK or use REST API to pass the control plane token. For more information, see Score data using the key or token.

Prepare a user identity

You need a user identity to perform control plane operations (that is, CRUD operations) and data plane operations (that is, send scoring requests) on the online endpoint. You can use the same user identity or different user identities for the control plane and data plane operations. In this article, you use the same user identity for both control plane and data plane operations.

To create a user identity under Microsoft Entra ID, see Set up authentication. You'll need the identity ID later.

Assign permissions to the identity

In this section, you assign permissions to the user identity that you use for interacting with the endpoint. You begin by using either a built-in role or by creating a custom role. Thereafter, you assign the role to your user identity.

Use a built-in role

The AzureML Data Scientist built-in role can be used to manage and use endpoints and deployments and it uses wildcards to include the following control plane RBAC actions:

  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action
  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action

and to include the following data plane RBAC action:

  • Microsoft.MachineLearningServices/workspaces/onlineEndpoints/score/action

Optionally, the Azure Machine Learning Workspace Connection Secrets Reader built-in role can be used to access secrets from workspace connections and it include the following control plane RBAC actions:

  • Microsoft.MachineLearningServices/workspaces/connections/listsecrets/action
  • Microsoft.MachineLearningServices/workspaces/metadata/secrets/read

If you use these built-in roles, there's no action needed at this step.

(Optional) Create a custom role

You can skip this step if you're using built-in roles or other pre-made custom roles.

  1. Define the scope and actions for custom roles by creating JSON definitions of the roles. For example, the following role definition allows the user to CRUD an online endpoint, under a specified workspace.

    custom-role-for-control-plane.json:

    {
    "Name": "Custom role for control plane operations - online endpoint",
    "IsCustom": true,
    "Description": "Can CRUD against online endpoints.",
    "Actions": [
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/delete",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/read",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/token/action",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/listKeys/action",
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/regenerateKeys/action"
    ],
    "NotActions": [
    ],
    "AssignableScopes": [
        "/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>"
    ]
}

    The following role definition allows the user to send scoring requests to an online endpoint, under a specified workspace.

    custom-role-for-scoring.json:

    {
    "Name": "Custom role for scoring - online endpoint",
    "IsCustom": true,
    "Description": "Can score against online endpoints.",
    "Actions": [
        "Microsoft.MachineLearningServices/workspaces/onlineEndpoints/*/action"
    ],
    "NotActions": [
    ],
    "AssignableScopes": [
        "/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>"
    ]
}

  2. Use the JSON definitions to create custom roles:

    az role definition create --role-definition custom-role-for-control-plane.json --subscription <subscriptionId>

az role definition create --role-definition custom-role-for-scoring.json --subscription <subscriptionId>

    Note

    To create custom roles, you need one of three roles:

    • owner
    • user access administrator
    • a custom role with Microsoft.Authorization/roleDefinitions/write permission (to create/update/delete custom roles) and Microsoft.Authorization/roleDefinitions/read permission (to view custom roles).

    For more information on creating custom roles, see Azure custom roles.

  3. Verify the role definition:

    az role definition list --custom-role-only -o table

az role definition list -n "Custom role for control plane operations - online endpoint"
az role definition list -n "Custom role for scoring - online endpoint"

export role_definition_id1=`(az role definition list -n "Custom role for control plane operations - online endpoint" --query "[0].id" | tr -d '"')`

export role_definition_id2=`(az role definition list -n "Custom role for scoring - online endpoint" --query "[0].id" | tr -d '"')`

Assign the role to the identity

  1. If you're using the AzureML Data Scientist built-in role, use the following code to assign the role to your user identity.

    az role assignment create --assignee <identityId> --role "AzureML Data Scientist" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

  2. Optionally, if you're using the Azure Machine Learning Workspace Connection Secrets Reader built-in role, use the following code to assign the role to your user identity.

    az role assignment create --assignee <identityId> --role "Azure Machine Learning Workspace Connection Secrets Reader" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

  3. If you're using a custom role, use the following code to assign the role to your user identity.

    az role assignment create --assignee <identityId> --role "Custom role for control plane operations - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

az role assignment create --assignee <identityId> --role "Custom role for scoring - online endpoint" --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

    Note

    To assign custom roles to the user identity, you need one of three roles:

    • owner
    • user access administrator
    • a custom role that allows Microsoft.Authorization/roleAssignments/write permission (to assign custom roles) and Microsoft.Authorization/roleAssignments/read (to view role assignments).

    For more information on the different Azure roles and their permissions, see Azure roles and Assigning Azure roles using Azure Portal.

  4. Confirm the role assignment:

    az role assignment list --scope /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.MachineLearningServices/workspaces/<workspaceName>

Get the Microsoft Entra token for control plane operations

Perform this step if you plan to perform control plane operations with REST API, which will directly use the token.

If you plan to use other ways such as Azure Machine Learning CLI (v2), Python SDK (v2), or the Azure Machine Learning studio, you don't need to get the Microsoft Entra token manually. Rather, during sign in, your user identity would already be authenticated, and the token would automatically be retrieved and passed for you.

You can retrieve the Microsoft Entra token for control plane operations from the Azure resource endpoint: https://management.azure.com.

  1. Sign in to Azure.

    az login

  2. If you want to use a specific identity, use the following code to sign in with the identity:

    az login --identity --username <identityId>

  3. Use this context to get the token.

    export CONTROL_PLANE_TOKEN=`(az account get-access-token --resource https://management.azure.com --query accessToken | tr -d '"')`

(Optional) Verify the resource endpoint and client ID for Microsoft Entra token

After you retrieve the Microsoft Entra token, you can verify that the token is for the right Azure resource endpoint management.azure.com and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:

{
    "aud": "https://management.azure.com",
    "oid": "<your-object-id>"
}

Create an endpoint

The following example creates the endpoint with a system-assigned identity (SAI) as the endpoint identity. The SAI is the default identity type of the managed identity for endpoints. Some basic roles are automatically assigned for the SAI. For more information on role assignment for a system-assigned identity, see Automatic role assignment for endpoint identity.

The CLI doesn't require you to explicitly provide the control plane token. Instead, the CLI authenticates you during sign in, and the token is automatically retrieved and passed for you.

  1. Create an endpoint definition YAML file.

    endpoint.yml:

    $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: my-endpoint
auth_mode: aad_token

  2. You can replace auth_mode with key for key auth, or aml_token for Azure Machine Learning token auth. In this example, you use aad_token for Microsoft Entra token auth.

    az ml online-endpoint create -f endpoint.yml

  3. Check the endpoint's status:

    az ml online-endpoint show -n my-endpoint

  4. If you want to override auth_mode (for example, to aad_token) when creating an endpoint, run the following code:

    az ml online-endpoint create -n my-endpoint --auth_mode aad_token

  5. If you want to update the existing endpoint and specify auth_mode (for example, to aad_token), run the following code:

    az ml online-endpoint update -n my-endpoint --set auth_mode=aad_token

Create a deployment

To create a deployment, see Deploy an ML model with an online endpoint or Use REST to deploy an model as an online endpoint. There's no difference in how you create deployments for different auth modes.

The following code is an example of how to create a deployment. For more information on deploying online endpoints, see Deploy an ML model with an online endpoint (via CLI)

  1. Create a deployment definition YAML file.

    blue-deployment.yml:

    $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
name: blue
endpoint_name: my-aad-auth-endp1
model:
  path: ../../model-1/model/
code_configuration:
  code: ../../model-1/onlinescoring/
  scoring_script: score.py
environment: 
  conda_file: ../../model-1/environment/conda.yml
  image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest
instance_type: Standard_DS3_v2
instance_count: 1

  2. Create the deployment using the YAML file. For this example, set all traffic to the new deployment.

    az ml online-deployment create -f blue-deployment.yml --all-traffic

Get the scoring URI for the endpoint

If you plan to use the CLI to invoke the endpoint, you're not required to get the scoring URI explicitly, as the CLI handles it for you. However, you can still use the CLI to get the scoring URI so that you can use it with other channels, such as REST API.

scoringUri=$(az ml online-endpoint show -n my-endpoint --query "scoring_uri")

Get the key or token for data plane operations

A key or token can be used for data plane operations, even though the process of getting the key or token is a control plane operation. In other words, you use a control plane token to get the key or token that you later use to perform your data plane operations.

Getting the key or Azure Machine Learning token requires that the correct role is assigned to the user identity that is requesting it, as described in authorization for control plane operations. The user identity doesn't need any extra roles to get the Microsoft Entra token.

Key or Azure Machine Learning token

If you plan to use the CLI to invoke the endpoint, and if the endpoint is set up to use an auth mode of key or Azure Machine Learning token (aml_token), you're not required to get the data plane token explicitly, as the CLI handles it for you. However, you can still use the CLI to get the data plane token so that you can use it with other channels, such as REST API.

To get the key or Azure Machine Learning token (aml_token), use the az ml online-endpoint get-credentials command. This command returns a JSON document that contains the key or Azure Machine Learning token.

Keys are returned in the primaryKey and secondaryKey fields. The following example shows how to use the --query parameter to return only the primary key:

export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query primaryKey)

Azure Machine Learning Tokens are returned in the accessToken field:

export DATA_PLANE_TOKEN=$(az ml online-endpoint get-credentials -n $ENDPOINT_NAME -g $RESOURCE_GROUP -w $WORKSPACE_NAME -o tsv --query accessToken)

Also, the expiryTimeUtc and refreshAfterTimeUtc fields contain the token expiration and refresh times.

Microsoft Entra token

To get the Microsoft Entra token (aad_token) using CLI, use the az account get-access-token command. This command returns a JSON document that contains the Microsoft Entra token.

Microsoft Entra token is returned in the accessToken field:

export DATA_PLANE_TOKEN=`(az account get-access-token --resource https://ml.azure.com --query accessToken | tr -d '"')`

Note

  • The CLI ml extension doesn't support getting the Microsoft Entra token. Use az account get-access-token instead, as described in the previous code.
  • The token for data plane operations is retrieved from the Azure resource endpoint ml.azure.com instead of management.azure.com, unlike the token for control plane operations.

Verify the resource endpoint and client ID for the Microsoft Entra token

After getting the Entra token, you can verify that the token is for the right Azure resource endpoint ml.azure.com and the right client ID by decoding the token via jwt.ms, which will return a json response with the following information:

{
    "aud": "https://ml.azure.com",
    "oid": "<your-object-id>"
}

Score data using the key or token

Key or Azure Machine Learning token

You can use az ml online-endpoint invoke for endpoints with a key or Azure Machine Learning token. The CLI handles the key or Azure Machine Learning token automatically so you don't need to pass it explicitly.

az ml online-endpoint invoke -n my-endpoint -r request.json

Microsoft Entra token

Using az ml online-endpoint invoke for endpoints with a Microsoft Entra token isn't supported. Use REST API instead, and use the endpoint's scoring URI to invoke the endpoint.

Log and monitor traffic

To enable traffic logging in the diagnostics settings for the endpoint, follow the steps in How to enable/disable logs.

If the diagnostic setting is enabled, you can check the AmlOnlineEndpointTrafficLogs table to see the auth mode and user identity.

Related content