Issue with authenticating API management with Azure OpenAI services using managed identity

Question

Hello,

We are trying to connect API management services with one or multiple model deployments on Azure OpenAI services. Basically we are trying to do some version of what is shown in this repo (and this blog post).

Following the instructions, we have model deployments in multiple regions (a mix of gpt models) with multiple endpoints and multiple keys. We are trying to use API management as an unique endpoint. We are also leveraging the subscription capabilities of API management in order to handle one unique API key.

The APIs are configured in API management for each model, with the appropriate openapi specification file from the Azure reference repository. The selection of the backend service in API management is defined by a policy that seems to be working, based on the query logs. The requests are forwarded to the appropriate Azure OpenAI endpoint.

The problem is with the authentication using managed identity between API management and Azure OpenAI services. We created an user assigned identity, assigned the roles "Cognitive Services User" and "Cognitive Services OpenAI User" to that identity. This identity is then assigned to the API management instance we created. In the API management policy, we are using the "authentication-managed-identity" directive in order to retrieve an access token for that identity for the resource "https://cognitiveservices.azure.com", passing the identity's client id in order to ensure we are using Entra ID.

The resulting token seems valid but when forwarded to the Azure OpenAI service, the backend request fails with a status 400 response :

{
	"error": {
		"code": "Request is badly formated",
		"message": "Resource Id is badly formed: NA"
	}
}

We tried using that token directly with Azure OpenAI service without going through API management, and it also breaks, whereas using the "api-key" header as normal works for the same request.

Due to the lack of additional information on why the request failed, we need some help. Is there anything we forgot ? Or is there some sort of incompatibility between the managed identity and Azure OpenAI services ?

Thank you for your help

Answer 1

Hello twin,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

Problem

Based on your questions, it appears you are setting up Azure API Management (APIM) to handle requests for multiple Azure OpenAI model deployments using a managed identity for authentication. Despite following the setup process, you are encountering a 400 error indicating a malformed resource ID when using a token obtained via the managed identity. This token fails both through APIM and in direct requests, while the API key works correctly.

Scenario

The organization has several Azure OpenAI model deployments across different regions. They want to streamline access to these models by using Azure API Management as a single endpoint. By leveraging APIM's subscription capabilities, they aim to manage access with a single API key. The plan is to use a managed identity for authentication between APIM and the Azure OpenAI service. However, despite setting up the managed identity and assigning the necessary roles correctly, authentication fails with a 400 status code indicating a malformed resource ID.

Solution

The lack of detailed error information makes troubleshooting difficult. Here are the steps to troubleshoot the authentication issue between Azure API Management (APIM) and Azure OpenAI services using a managed identity:

• Confirm that the resource URI for which you are requesting the token is correct. For Azure OpenAI services.
• Make sure the user-assigned managed identity has the correct roles (Cognitive Services User and Cognitive Services OpenAI User) assigned to it.
• Verify the token retrieval process in APIM. Ensure that the token is indeed being requested for the correct managed identity client ID.
• Double-check the APIM inbound policy configuration.

Finally

Enable detailed logging in APIM to capture more information about the failed requests

References

Source: Using Managed Identity to Access Azure OpenAI Service. Accessed, 5/17/2024.

Source: How to configure Azure OpenAI Service with managed identities. Accessed, 5/17/2024.

Source: Troubleshoot Azure Automation managed identity issues. Accessed, 5/17/2024.

Accept Answer

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful ** so that others in the community facing similar issues can easily find the solution.

Best Regards,

Sina Salam

Answer 2

Solution:

The problem lies in the use of regional endpoints instead of custom endpoints as the backend url in API Management. Please ensure that when you specify the backend URL, it is not a regional endpoint but a custom endpoint.

Example:

Wrong backend url endpoint: https://<region>.api.cognitive.microsoft.com

Correct backend url endpoint: https://<custom-name>.openai.azure.com

Note:

To find the Azure OpenAI service endpoint, you can follow these steps:

1. Go to your Azure openAI service resource in the Azure portal.
2. In the Keys & Endpoint section, copy your endpoint.
3. You can also find the value in the Azure OpenAI Studio > Playground > Code View.