Import an Azure OpenAI API

APPLIES TO: All API Management tiers

This article shows two options to import an Azure OpenAI Service API into an Azure API Management instance as a REST API:

• Import an Azure OpenAI API directly from Azure OpenAI Service (recommended)
• Download and add the OpenAPI specification for Azure OpenAI and add it to API Management as an OpenAPI API.

Prerequisites

• An existing API Management instance. Create one if you haven't already.

• An Azure OpenAI resource with a model deployed. For more information about model deployment, see the resource deployment guide.

  Make a note of the ID (name) of the deployment. You'll need it when you test the imported API in API Management.

• Permissions to grant access to the Azure OpenAI resource from the API Management instance.

Option 1. Import API from Azure OpenAI Service

You can import an Azure OpenAI API directly from Azure OpenAI Service to API Management.

Note

Currently, this feature isn't available in workspaces.

When you import the API, API Management automatically configures:

• Operations for each of the Azure OpenAI REST API endpoints.
• A system-assigned identity with the necessary permissions to access the Azure OpenAI resource.
• A backend resource and set-backend-service policy that direct API requests to the Azure OpenAI Service endpoint.
• An authentication-managed-identity policy that can authenticate to the Azure OpenAI resource using the instance's system-assigned identity.
• (optionally) Policies to help you monitor and manage token consumption by the Azure OpenAI API.

To import an Azure OpenAI API to API Management:

1. In the Azure portal, navigate to your API Management instance.

2. In the left menu, under APIs, select APIs > + Add API.

3. Under Create from Azure resource, select Azure OpenAI Service.

   

4. On the Basics tab:

   1. Select the Azure OpenAI resource that you want to import.

   2. Optionally select an Azure OpenAI API version. If you don't select one, the latest production-ready REST API version is used by default.

   3. Enter a Display name and optional Description for the API.

   4. In Base URL, append a path that your API Management instance uses to access the Azure OpenAI API endpoints. If you enable Ensure OpenAI SDK compatibility (recommended), /openai is automatically appended to the base URL.

      For example, if your API Management gateway endpoint is https://contoso.azure-api.net, set a Base URL similar to https://contoso.azure-api.net/my-openai-api/openai.

   5. Optionally select one or more products to associate with the API. Select Next.

5. On the Policies tab, optionally enable policies to monitor and manage Azure OpenAI API token consumption. If selected, enter settings or accept defaults that define the azure-openai-token-limit and azure-openai-emit-token-metric policies for your API. You can also set or update the policy configuration later. Select Review + Create.

6. After settings are validated, select Create.

Option 2. Add an OpenAPI specification to API Management

Alternatively, manually download the OpenAPI specification for the Azure OpenAI REST API and add it to API Management as an OpenAPI API.

Download the OpenAPI specification

Download the OpenAPI specification for the Azure OpenAI REST API, such as the 2024-02-01 GA version.

1. In a text editor, open the specification file that you downloaded.

2. In the servers element in the specification, substitute the name of your Azure OpenAI Service endpoint in the placeholder values of url and default endpoint in the specification. For example, if your Azure OpenAI Service endpoint is contoso.openai.azure.com, update the servers element with the following values:

   • url: https://contoso.openai.azure.com/openai
   • default endpoint: contoso.openai.azure.com
   JSON

   [...]
   "servers": [
       {
         "url": "https://contoso.openai.azure.com/openai",
         "variables": {
           "endpoint": {
             "default": "contoso.openai.azure.com"
           }
         }
       }
     ],
   [...]
   

3. Make a note of the value of the API version in the specification. You'll need it to test the API. Example: 2024-02-01.

Add OpenAPI specification to API Management

1. In the Azure portal, navigate to your API Management instance.
2. In the left menu, select APIs > + Add API.
3. Under Define a new API, select OpenAPI. Enter a Display name and Name for the API.
4. Enter an API URL suffix ending in /openai to access the Azure OpenAI API endpoints in your API Management instance. Example: my-openai-api/openai.
5. Select Create.

The API is imported and displays operations from the OpenAPI specification.

Configure authentication to Azure OpenAI API

To authenticate to the Azure OpenAI API, you supply an API key or a managed identity. If you imported the Azure OpenAI API directly to your API Management instance, authentication using the API Management instance's managed identity is automatically configured.

If you added the Azure OpenAI API from its OpenAPI specification, you need to configure authentication. For more information about configuring authentication using API Management policies, see Authenticate and authorize to Azure OpenAI API.

Test the Azure OpenAI API

To ensure that your Azure OpenAI API is working as expected, test it in the API Management test console. You need to supply a model deployment ID (name) configured in the Azure OpenAI resource to test the API.

1. Select the API you created in the previous step.

2. Select the Test tab.

3. Select an operation that's compatible with the model you deployed in the Azure OpenAI resource. The page displays fields for parameters and headers.

4. In Template parameters, enter the following values:

   • deployment-id - the ID of a deployment in the Azure OpenAI service
   • api-version - a valid Azure OpenAI API version, such as the API version you selected when you imported the API.

5. Enter other parameters and headers as needed. Depending on the operation, you may need to configure or update a Request body.

   Note

   In the test console, API Management automatically populates an Ocp-Apim-Subscription-Key header, and configures the subscription key of the built-in all-access subscription. This key enables access to every API in the API Management instance. Optionally display the Ocp-Apim-Subscription-Key header by selecting the "eye" icon next to the HTTP Request.

6. Select Send.

   When the test is successful, the backend responds with a successful HTTP response code and some data. Appended to the response is token usage data to help you monitor and manage your Azure OpenAI API consumption.

Caching policies for Azure OpenAI APIs

In addition to the azure-openai-token-limit and azure-openai-emit-token-metric policies that you can configure when importing an Azure OpenAI Service API, API Management provides the following caching policies to help you optimize performance and reduce latency for Azure OpenAI APIs:

• azure-openai-semantic-cache-store
• azure-openai-semantic-cache-lookup

Related content

• Generative AI gateway capabilities in Azure API Management
• API Management policy reference

Related topics

• API import limitations
• Import an OpenAPI specification
• Import a SOAP API
• Import a SOAP API and convert to REST
• Import an App Service API
• Import a Container App API
• Import a WebSocket API
• Import a GraphQL API
• Import a GraphQL schema and set up field resolvers
• Import an Azure Function App
• Import an Azure Logic App
• Import a Service Fabric service
• Import an Azure OpenAI API
• Import an OData API
• Import SAP OData metadata
• Import a gRPC API
• Edit an API