AML skill in an Azure AI Search enrichment pipeline

Article
08/28/2024

Important

Support for indexer connections to the Azure AI Studio model catalog is in public preview under supplemental terms of use. Preview REST APIs support this skill.

The AML skill allows you to extend AI enrichment with a custom Azure Machine Learning (AML) model. Once an AML model is trained and deployed, an AML skill integrates it into AI enrichment.

Like other built-in skills, an AML skill has inputs and outputs. The inputs are sent to your deployed AML online endpoint as a JSON object, which outputs a JSON payload as a response along with a success status code. Your data is processed in the Geo where your model is deployed. The response is expected to have the outputs specified by your AML skill. Any other response is considered an error and no enrichments are performed.

The AML skill can be called with the 2024-07-01 stable API version or the 2024-05-01-preview API version for connections to the model catalog in Azure AI Studio.

Starting in 2024-05-01-preview REST API and in the Azure portal (which also targets the 2024-05-01-preview), Azure AI Search introduced the Azure AI Studio model catalog vectorizer for query time connections to the model catalog in Azure AI Studio. If you want to use that vectorizer for queries, the AML skill is the indexing counterpart for generating embeddings using a model in the Azure AI Studio model catalog.

During indexing, the AML skill can connect to the model catalog to generate vectors for the index. At query time, queries can use a vectorizer to connect to the same model to vectorize text strings for a vector query. In this workflow, the AML skill and the model catalog vectorizer should be used together so that you're using the same embedding model for both indexing and queries. See How to implement integrated vectorization using models from Azure AI Studio for details on this workflow.

Note

The indexer will retry twice for certain standard HTTP status codes returned from the AML online endpoint. These HTTP status codes are:

503 Service Unavailable
429 Too Many Requests

Prerequisites

An AML workspace
An Online endpoints (real-time) in this workspace.

@odata.type

Microsoft.Skills.Custom.AmlSkill

Skill parameters

Parameters are case-sensitive. Which parameters you choose to use depends on what authentication your AML online endpoint requires, if any

Parameter name	Description
`uri`	(Required for key authentication) The scoring URI of the AML online endpoint to which the JSON payload is sent. Only the https URI scheme is allowed.
`key`	(Required for key authentication) The key for the AML online endpoint.
`resourceId`	(Required for token authentication). The Azure Resource Manager resource ID of the AML online endpoint. It should be in the format subscriptions/{guid}/resourceGroups/{resource-group-name}/Microsoft.MachineLearningServices/workspaces/{workspace-name}/onlineendpoints/{endpoint_name}.
`region`	(Optional for token authentication). The region the AML online endpoint is deployed in.
`timeout`	(Optional) When specified, indicates the timeout for the http client making the API call. It must be formatted as an XSD "dayTimeDuration" value (a restricted subset of an ISO 8601 duration value). For example, `PT60S` for 60 seconds. If not set, a default value of 30 seconds is chosen. The timeout can be set to a maximum of 230 seconds and a minimum of 1 second.
`degreeOfParallelism`	(Optional) When specified, indicates the number of calls the indexer makes in parallel to the endpoint you have provided. You can decrease this value if your endpoint is failing under too high of a request load. You can raise it if your endpoint is able to accept more requests and you would like an increase in the performance of the indexer. If not set, a default value of 5 is used. The degreeOfParallelism can be set to a maximum of 10 and a minimum of 1.

What skill parameters to use

Which AML skill parameters are required depends on what authentication your AML online endpoint uses, if any. AML online endpoints provide two authentication options:

Key-Based Authentication. A static key is provided to authenticate scoring requests from AML skills
- Use the uri and key parameters
Token-Based Authentication. The AML online endpoint is deployed using token based authentication. The Azure AI Search service's managed identity must be enabled. The AML skill then uses the service's managed identity to authenticate against the AML online endpoint, with no static keys required. The identity must be assigned owner or contributor role.
- Use the resourceId parameter.
- If the search service is in a different region from the AML workspace, use the region parameter to set the region the AML online endpoint was deployed in

Skill inputs

There are no "predefined" inputs for this skill. You can choose one or more fields that would be already available at the time of this skill's execution as inputs and the JSON payload sent to the AML online endpoint will have different fields.

Skill outputs

There are no "predefined" outputs for this skill. Depending on the response your AML online endpoint returns, add output fields so that they can be picked up from the JSON response.

Sample definition

  {
    "@odata.type": "#Microsoft.Skills.Custom.AmlSkill",
    "description": "A sample model that detects the language of sentence",
    "uri": "https://contoso.count-things.com/score",
    "context": "/document",
    "inputs": [
      {
        "name": "text",
        "source": "/document/content"
      }
    ],
    "outputs": [
      {
        "name": "detected_language_code"
      }
    ]
  }

Sample input JSON structure

This JSON structure represents the payload that is sent to your AML online endpoint. The top-level fields of the structure correspond to the "names" specified in the inputs section of the skill definition. The values of those fields are from the source of those fields (which could be from a field in the document, or potentially from another skill)

{
  "text": "Este es un contrato en Inglés"
}

Sample output JSON structure

The output corresponds to the response returned from your AML online endpoint. The AML online endpoint should only return a JSON payload (verified by looking at the Content-Type response header) and should be an object where the fields are enrichments matching the "names" in the output and whose value is considered the enrichment.

{
    "detected_language_code": "es"
}

Inline shaping sample definition

  {
    "@odata.type": "#Microsoft.Skills.Custom.AmlSkill",
    "description": "A sample model that detects the language of sentence",
    "uri": "https://contoso.count-things.com/score",
    "context": "/document",
    "inputs": [
      {
        "name": "shapedText",
        "sourceContext": "/document",
        "inputs": [
            {
              "name": "content",
              "source": "/document/content"
            }
        ]
      }
    ],
    "outputs": [
      {
        "name": "detected_language_code"
      }
    ]
  }

Inline shaping input JSON structure

{
  "shapedText": { "content": "Este es un contrato en Inglés" }
}

Inline shaping sample output JSON structure

{
    "detected_language_code": "es"
}

Error cases

In addition to your AML being unavailable or sending out nonsuccessful status codes, the following are considered erroneous cases:

If the AML online endpoint returns a success status code but the response indicates that it isn't application/json, then the response is considered invalid and no enrichments are performed.
If the AML online endpoint returns invalid json

For cases when the AML online endpoint is unavailable or returns an HTTP error, a friendly error with any available details about the HTTP error will be added to the indexer execution history.

Share via