Edit

Share via

Deploy models with REST

  • Article

This article describes how to use the Azure Machine Learning REST API to deploy models by using online endpoints. Online endpoints allow you to deploy your model without having to create and manage the underlying infrastructure and Kubernetes clusters. The following procedures demonstrate how to create an online endpoint and deployment and validate the endpoint by invoking it.

There are many ways to create an Azure Machine Learning online endpoint. You can use the Azure CLI, the Azure Machine Learning studio, or the REST API. The REST API uses standard HTTP verbs to create, retrieve, update, and delete resources. It works with any language or tool that can make HTTP requests. The straightforward structure of the REST API makes it a good choice in scripting environments and for machine learning operations automation.

Prerequisites

Set endpoint name

Endpoint names must be unique at the Azure region level. An endpoint name such as my-endpoint must be the only endpoint with that name within a specified region.

Create a unique endpoint name by calling the RANDOM utility, which adds a random number as a suffix to the value endpt-rest:

export ENDPOINT_NAME=endpt-rest-`echo $RANDOM`

Create machine learning assets

To prepare for the deployment, set up your Azure Machine Learning assets and configure your job. You register the assets required for deployment, including the model, code, and environment.

Tip

The REST API calls in the following procedures use $SUBSCRIPTION_ID, $RESOURCE_GROUP, $LOCATION (region), and Azure Machine Learning $WORKSPACE as placeholders for some arguments. When you implement the code for your deployment, replace the argument placeholders with your specific deployment values.

Administrative REST requests a service principal authentication token. When you implement the code for your deployment, replace instances of the $TOKEN placeholder with the service principal token for your deployment. You can retrieve this token with the following command:

response=$(curl -H "Content-Length: 0" --location --request POST "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/onlineEndpoints/$ENDPOINT_NAME/token?api-version=$API_VERSION" \
--header "Authorization: Bearer $TOKEN")
accessToken=$(echo $response | jq -r '.accessToken')

The service provider uses the api-version argument to ensure compatibility. The api-version argument varies from service to service.

Set the API_version variable to accommodate future versions:

API_VERSION="2022-05-01"

Get storage account details

To register the model and code, you need to first upload these items to an Azure Storage account. The details of the Azure Storage account are available in the data store. In this example, you get the default data store and Azure Storage account for your workspace. Query your workspace with a GET request to get a JSON file with the information.

You can use the jq tool to parse the JSON result and get the required values. You can also use the Azure portal to find the same information:

# Get values for storage account
response=$(curl --location --request GET "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/datastores?api-version=$API_VERSION&isDefault=true" \
--header "Authorization: Bearer $TOKEN")
AZUREML_DEFAULT_DATASTORE=$(echo $response | jq -r '.value[0].name')
AZUREML_DEFAULT_CONTAINER=$(echo $response | jq -r '.value[0].properties.containerName')
export AZURE_STORAGE_ACCOUNT=$(echo $response | jq -r '.value[0].properties.accountName')

Upload and register code

Now that you have the data store, you can upload the scoring script. Use the Azure Storage CLI to upload a blob into your default container:

az storage blob upload-batch -d $AZUREML_DEFAULT_CONTAINER/score -s endpoints/online/model-1/onlinescoring

Tip

You can use other methods to complete the upload, such as the Azure portal or Azure Storage Explorer.

After you upload your code, you can specify your code with a PUT request and refer to the data store with the datastoreId identifier:

curl --location --request PUT "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/codes/score-sklearn/versions/1?api-version=$API_VERSION" \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/json" \
--data-raw "{
  \"properties\": {
    \"codeUri\": \"https://$AZURE_STORAGE_ACCOUNT.blob.core.windows.net/$AZUREML_DEFAULT_CONTAINER/score\"
  }
}"

Upload and register model

Upload the model files with a similar REST API call:

az storage blob upload-batch -d $AZUREML_DEFAULT_CONTAINER/model -s endpoints/online/model-1/model

After the upload completes, register the model:

curl --location --request PUT "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/models/sklearn/versions/1?api-version=$API_VERSION" \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/json" \
--data-raw "{
    \"properties\": {
        \"modelUri\":\"azureml://subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/workspaces/$WORKSPACE/datastores/$AZUREML_DEFAULT_DATASTORE/paths/model\"
    }
}"

Create environment

The deployment needs to run in an environment that has the required dependencies. Create the environment with a PUT request. Use a Docker image from Microsoft Container Registry. You can configure the Docker image with the docker command and add conda dependencies with the condaFile command.

The following code reads the contents of a Conda environment (YAML file) into an environment variable:

ENV_VERSION=$RANDOM
curl --location --request PUT "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/environments/sklearn-env/versions/$ENV_VERSION?api-version=$API_VERSION" \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/json" \
--data-raw "{
    \"properties\":{
        \"condaFile\": \"$CONDA_FILE\",
        \"image\": \"mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20210727.v1\"
    }
}"

Create endpoint

Create the online endpoint:

response=$(curl --location --request PUT "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/onlineEndpoints/$ENDPOINT_NAME?api-version=$API_VERSION" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" \
--data-raw "{
    \"identity\": {
       \"type\": \"systemAssigned\"
    },
    \"properties\": {
        \"authMode\": \"AMLToken\"
    },
    \"location\": \"$LOCATION\"
}")

Create deployment

Create a deployment under the endpoint:

response=$(curl --location --request PUT "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/onlineEndpoints/$ENDPOINT_NAME/deployments/blue?api-version=$API_VERSION" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" \
--data-raw "{
    \"location\": \"$LOCATION\",
    \"sku\": {
        \"capacity\": 1,
        \"name\": \"Standard_DS2_v2\"
    },
    \"properties\": {
        \"endpointComputeType\": \"Managed\",
        \"scaleSettings\": {
            \"scaleType\": \"Default\"
        },
        \"model\": \"/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/models/sklearn/versions/1\",
        \"codeConfiguration\": {
            \"codeId\": \"/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/codes/score-sklearn/versions/1\",
            \"scoringScript\": \"score.py\"
        },
        \"environmentId\": \"/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/environments/sklearn-env/versions/$ENV_VERSION\"
    }
}")

Invoke endpoint to score data with model

You need the scoring URI and access token to invoke the deployment endpoint.

First, get the scoring URI:

response=$(curl --location --request GET "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/onlineEndpoints/$ENDPOINT_NAME?api-version=$API_VERSION" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN")

scoringUri=$(echo $response | jq -r '.properties.scoringUri')

Next, get the endpoint access token:

response=$(curl -H "Content-Length: 0" --location --request POST "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/onlineEndpoints/$ENDPOINT_NAME/token?api-version=$API_VERSION" \
--header "Authorization: Bearer $TOKEN")
accessToken=$(echo $response | jq -r '.accessToken')

Finally, invoke the endpoint by using the curl utility:

curl --location --request POST $scoringUri \
--header "Authorization: Bearer $accessToken" \
--header "Content-Type: application/json" \
--data-raw @endpoints/online/model-1/sample-request.json

Check deployment logs

Check the deployment logs:

curl --location --request POST "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/onlineEndpoints/$ENDPOINT_NAME/deployments/blue/getLogs?api-version=$API_VERSION" \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/json" \
--data-raw "{ \"tail\": 100 }"

Delete endpoint

If you aren't going to use the deployment further, delete the resources.

Run the following command, which deletes the endpoint and all underlying deployments:

curl --location --request DELETE "https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.MachineLearningServices/workspaces/$WORKSPACE/onlineEndpoints/$ENDPOINT_NAME?api-version=$API_VERSION" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $TOKEN" || true

Related content