Azure API Management
An Azure service that provides a hybrid, multi-cloud management platform for APIs.
2,447 questions
Getting error while configuraing semantic cahce in apim
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 17%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERS%3C/text%3E%3C/svg%3E)
Rajnish Soni
60
Reputation points
Hello Team,
I am trying to configure azure APIM with semantic caching and getting error message in trace messages.
can you please help to fix this.
my backend custom url is
https://my-openai.openai.azure.com
i am getting below error:
azure-openai-semantic-cache-lookup (185.049 ms)
{
"response": {
"status": {
"code": 404,
"reason": "Resource Not Found"
},
azure-openai-semantic-cache-lookup (0.053 ms)
{
"message": "Failed to get embeddings from the backend endpoint. Skipping caching. Status code: NotFound."
}
in the forward request, apim is appending the api version automatically which is wrong.
"url": "https://my-openai.openai.azure.com/?api-version=2024-02-01",
Azure API Management
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 77%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Sai Prabhu Naveen Parimi 2,265 Reputation points Microsoft External Staff Moderator2025-03-27T09:09:46.6133333+00:00 Based on your trace logs, it looks like APIM is appending
?api-version=2024-02-01directly to the root URL, which may not be correct. Azure OpenAI API typically expects a specific path in the request.To help troubleshoot, could you confirm the following?
- What is the expected full request URL for your API call? (It should typically be something like:
https://my-openai.openai.azure.com/openai/deployments/{deployment_id}/embeddings?api-version=2024-02-01 instead of justhttps://my-openai.openai.azure.com/?api-version=2024-02-01). - Are there any APIM policies (
rewrite-uri,set-query-parameter, etc.) that might be modifying the request? - Have you tested calling the API directly (outside of APIM) using Postman or cURL? If so, do you receive the same 404 error?
- Can you confirm that the deployment ID and any required parameters are included in the request?
Once we have this information, we can further investigate the issue and suggest possible fixes.
- What is the expected full request URL for your API call? (It should typically be something like:
-
Rajnish Soni 60 Reputation points
2025-03-27T10:06:17.2333333+00:00 this is the policy file i am using.
<inbound> <base /> <authentication-managed-identity resource="https://cognitiveservices.azure.com" output-token-variable-name="managed-id-access-token" ignore-error="false" /> <set-header name="Authorization" exists-action="override"> <value>@("Bearer " + (string)context.Variables["managed-id-access-token"])</value> </set-header> <set-backend-service backend-id="redisdemoapi-openai-endpoint" /> <authentication-managed-identity resource="https://cognitiveservices.azure.com/" /> <azure-openai-semantic-cache-lookup score-threshold="0.8" embeddings-backend-id="redisdemoapi-openai-endpoint" embeddings-backend-auth="system-assigned" ignore-system-messages="true" max-message-count="10"> <vary-by>@(context.Subscription.Id)</vary-by> </azure-openai-semantic-cache-lookup> </inbound>regarding the expected URL.
https://my-openai.openai.azure.com/openai/deployments/gpt-4_0613/chat/completions\?api-version=2024-05-01-previewi also called the openAI URL directly and able to get the 200 OK. All the required fileds deplloyment-id and api version is included in call. there are 2 section in the trace one is inbound and one is backend.
the inbound section logs is as below.
set-backend-service (0.046 ms) { "message": "Backend service URL was changed.", "oldBackendServiceUrl": "https://my-openai.openai.azure.com", "newBackendServiceUrl": "https://my-openai.openai.azure.com", "request": { "url": "https://my-openai.openai.azure.com/?api-version=2024-02-01" } } forward-request (0.228 ms) { "message": "Request is being forwarded to the backend service. Timeout set to 30 seconds", "request": { "method": "POST", "url": "https://my-openai.openai.azure.com/?api-version=2024-02-01",azure-openai-semantic-cache-lookup (194.338 ms) { "response": { "status": { "code": 404, "reason": "Resource Not Found" }, "headers": [ { "name": "Connection", "value": "keep-alive" },the backend section logs is as below.
set-backend-service (0.097 ms) { "message": "Backend service URL was changed.", "oldBackendServiceUrl": "https://my-openai.openai.azure.com", "newBackendServiceUrl": "https://my-openai.openai.azure.com", "request": { "url": "https://my-openai.openai.azure.com/openai/deployments/gpt-4_0613/chat/completions?api-version=2024-05-01-preview" } } forward-request (0.114 ms) { "message": "Request is being forwarded to the backend service. Timeout set to 300 seconds", "request": { "method": "POST", "url": "https://my-openai.openai.azure.com/openai/deployments/gpt-4_0613/chat/completions?api-version=2024-05-01-preview", "headers": [ { forward-request (1,632.287 ms) { "response": { "status": { "code": 200, "reason": "OK" },As you can see the backend section log clearly uses the correct URL and get the 200 response. my doubt is why the inbound log is showing 404 and getting wrong URL. the whole idea is to utilize the cache which is not happening now.
-
Sai Prabhu Naveen Parimi 2,265 Reputation points Microsoft External Staff Moderator
2025-03-27T12:17:52.39+00:00 Thanks for sharing the details. After reviewing the APIM policy, I noticed that the issue arises due to how the
set-backend-serviceis positioned after the cache lookup. This causes the cache lookup to hit an incorrect URL, which leads to the 404 error.To fix this, I’ve explicitly set the correct backend URL to ensure the request reaches OpenAI without APIM modifying the URL incorrectly. Here’s the corrected version of your policy:
<inbound> <base /> <authentication-managed-identity resource="https://cognitiveservices.azure.com" output-token-variable-name="managed-id-access-token" ignore-error="false" /> <set-header name="Authorization" exists-action="override"> <value>@("Bearer " + (string)context.Variables["managed-id-access-token"])</value> </set-header> <set-backend-service base-url="https://my-openai.openai.azure.com/openai/deployments/gpt-4_0613/chat/completions?api-version=2024-05-01-preview" /> <azure-openai-semantic-cache-lookup score-threshold="0.8" embeddings-backend-id="redisdemoapi-openai-endpoint" embeddings-backend-auth="system-assigned" ignore-system-messages="true" max-message-count="10"> <vary-by>@(context.Subscription.Id)</vary-by> </azure-openai-semantic-cache-lookup> </inbound>
This should resolve the 404 issue by preventing the wrong API version from being appended.
Please let me know if this resolves the issue or if further changes are required. I’m happy to assist further!
-
Rajnish Soni 60 Reputation points
2025-03-27T12:56:44.6266667+00:00 Hello Sai, I somehow managed to get the full url in forward request. only problem is it is appending the wrong api version every time. If we could add the correct api version then it will work i guess.
forward-request (0.215 ms) { "message": "Request is being forwarded to the backend service. Timeout set to 30 seconds", "request": { "method": "POST", "url": "https://my-openai.openai.azure.com/openai/deployments/gpt-4_0613/chat/completions?api-version=2024-02-01", -
Sai Prabhu Naveen Parimi 2,265 Reputation points Microsoft External Staff Moderator
2025-03-28T02:56:34.4166667+00:00 Thanks for the update. I see that APIM is still appending the wrong API version (
2024-02-01) instead of the latest version.To ensure the correct API version is always used, please update your APIM policy by explicitly setting the correct API version in the backend service URL:
<set-backend-service base-url="https://my-openai.openai.azure.com/openai/deployments/gpt-4_0613/chat/completions?api-version=2024-10-21" />This will prevent APIM from appending an incorrect version automatically.
For reference, you can check the latest supported API versions in the official Microsoft documentation:
Azure OpenAI API Versioning: API version lifecycle and deprecations
Azure OpenAI REST API Reference: Official API documentation
Let me know if this resolves the issue or if you need any further assistance.
-
Sai Prabhu Naveen Parimi 2,265 Reputation points Microsoft External Staff Moderator
2025-04-01T01:01:15.0466667+00:00 Just wanted to check if you had a chance to try the suggested fix. Let me know if it worked or if you need any further assistance.
Sign in to comment
Sign in to answer