Undesired 'Retrying request to /chat/completions' when making an API request to OpenAI in an Azure function.

Question

I'm using an Azure function in Python to make API requests to OpenAI and write the responses to a database. These requests can be long, so I switched to a premium plan. However, even though the long duration of the API call is expected and unavoidable, an undesired retry mechanism is being triggered.

This issue occurs in both HTTP trigger and Timer trigger versions of my function, despite being on a premium plan.

This retry is not linked to my code. When I execute my function locally (with or without a Docker image), I don't experience any retries, and my code completes as expected.

I suspect this might be related to Azure's built-in retry strategy for resilience, as mentioned in this Microsoft documentation, but I'm not certain, nor do I know how to modify this behavior.

I'd appreciate any insights on this issue.

Attached are screenshots of the logs for the exact same request executed from an Azure function: one running locally (on the right) and the other on Azure (on the left).

Answer 1

Hello @m_ft

Welcome to Microsoft QnA!

Hera are some possible solutions but you have to try :

1. set the timeout of your Azure Function. For example, in host.json:

json

{

"version": "2.0",

"functionTimeout": "00:60:00" // Premium has 30 min Default and you can set more, guaranteed to 60min

}

Read here for Limits and other details : https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale

1. Consider using Durable Functions as Orchestrator , Durable Functions is an extension of Azure Functions that lets you write stateful functions in a serverless compute environment. The extension lets you define stateful workflows by writing orchestrator functions and stateful entities by writing entity functions using the Azure Functions programming model. Behind the scenes, the extension manages state, checkpoints, and restarts for you, allowing you to focus on your business logic. https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=in-process%2Cnodejs-v3%2Cv1-model&pivots=csharp
2. Consider disabling retry policy :

{

"bindings": [

{

  "name": "myTimer",

  "type": "timerTrigger",

  "direction": "in",

  "schedule": "0 */5 * * * *",

  "retry": {"strategy": "fixedDelay", "maxRetryCount": 0, "delayInterval": "00:00:00"}

}

]

}

---

I hope this helps!

Kindly mark the answer as Accepted and Upvote in case it helped!

Regards