How to implement Throttling Backoff

Question

Hi, I'm trying to implement an additional backoff function to handle this error, how do I handle exceptions when writing code that applies labels using the MIP SDK with C#? Unfortunately, it takes too many requests to recreate the throttling limit on purpose, and since it's not an HTTP request, I can't directly handle the 429 status exception. How does the method respond when the throttling limit is exceeded. For example, CommitAsync or CreateFileHandlerAsync

Answer 1

Hi @정용우

Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!

To implement throttling backoff when using the MIP SDK in C#, you can follow the recommended practices for handling throttling errors. Although you may not receive a direct HTTP 429 status code, you should still be prepared to handle situations where your application exceeds service limits.

Implement Exponential Backoff: When you implement your app's error handling, use the HTTP error code 429 to detect the need for client-side throttling. If the request fails again with an HTTP 429 error code, you are still encountering an Azure service limit. Continue to use the recommended client-side throttling method, retrying the request until it succeeds.

Here is code that implements exponential backoff:

SecretClientOptions options = new SecretClientOptions()
    {
        Retry =
        {
            Delay= TimeSpan.FromSeconds(2),
            MaxDelay = TimeSpan.FromSeconds(16),
            MaxRetries = 5,
            Mode = RetryMode.Exponential
         }
    };
    var client = new SecretClient(new Uri("https://keyVaultName.vault.azure.net"), new DefaultAzureCredential(),options);
                                 
    //Retrieve Secret
    secret = client.GetSecret(secretName);

Monitor for Throttling: If your application encounters throttling, you should monitor the responses from the MIP SDK methods like CommitAsync or CreateFileHandlerAsync. If these methods indicate that a request cannot be processed due to throttling, you should pause your requests according to the backoff strategy.

Retry Logic: Implement retry logic that respects the backoff intervals. If a request fails, wait for the specified time before attempting the request again. You can use a loop to manage this retry logic, ensuring that you do not exceed the maximum number of retries.

For more details, please refer: How to throttle your app in response to service limits

I hope this information helps. Please do let us know if you have any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.