Undesired 'Retrying request to /chat/completions' when making an API request to OpenAI in an Azure function.

m_ft 5 Reputation points
2023-12-07T20:38:25.2233333+00:00

I'm using an Azure function in Python to make API requests to OpenAI and write the responses to a database. These requests can be long, so I switched to a premium plan. However, even though the long duration of the API call is expected and unavoidable, an undesired retry mechanism is being triggered.

This issue occurs in both HTTP trigger and Timer trigger versions of my function, despite being on a premium plan.

This retry is not linked to my code. When I execute my function locally (with or without a Docker image), I don't experience any retries, and my code completes as expected.

I suspect this might be related to Azure's built-in retry strategy for resilience, as mentioned in this Microsoft documentation, but I'm not certain, nor do I know how to modify this behavior.

I'd appreciate any insights on this issue.

Attached are screenshots of the logs for the exact same request executed from an Azure function: one running locally (on the right) and the other on Azure (on the left).

Screenshot 2023-12-08 at 11.09.34

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,942 questions
Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
4,115 questions
0 comments No comments
{count} vote

1 answer

Sort by: Most helpful
  1. Konstantinos Passadis 19,596 Reputation points MVP
    2023-12-07T21:06:53.63+00:00

    Hello @m_ft

    Welcome to Microsoft QnA!

    Hera are some possible solutions but you have to try :

    1. set the timeout of your Azure Function. For example, in host.json:

    json

    {

    "version": "2.0",

    "functionTimeout": "00:60:00" // Premium has 30 min Default and you can set more, guaranteed to 60min

    }

    Read here for Limits and other details : https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale

    1. Consider using Durable Functions as Orchestrator , Durable Functions is an extension of Azure Functions that lets you write stateful functions in a serverless compute environment. The extension lets you define stateful workflows by writing orchestrator functions and stateful entities by writing entity functions using the Azure Functions programming model. Behind the scenes, the extension manages state, checkpoints, and restarts for you, allowing you to focus on your business logic. https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=in-process%2Cnodejs-v3%2Cv1-model&pivots=csharp
    2. Consider disabling retry policy :

    {

    "bindings": [

    {
    
      "name": "myTimer",
    
      "type": "timerTrigger",
    
      "direction": "in",
    
      "schedule": "0 */5 * * * *",
    
      "retry": {"strategy": "fixedDelay", "maxRetryCount": 0, "delayInterval": "00:00:00"}
    
    }
    

    ]

    }


    I hope this helps!

    Kindly mark the answer as Accepted and Upvote in case it helped!

    Regards


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.