Server Error Countermeasures for FR

Question

Server Error Countermeasures for FR

ryota.matsuda 140

We have previously learned to make a request to raise the endpoint quota as a 429 error countermeasure.

But besides that, we think we need to deal with the 500th error as well.

To do so, I think we need to increase the redundancy of the endpoints. Is it necessary to take countermeasures at the level we provide to about 10,000 customers in the same region?

And if so, what are the best practice responses?

romungi-MSFT 48,911 Reputation points Microsoft Employee Moderator

2023-06-15T11:44:08.0333333+00:00
@ryota.matsuda All cognitive services APIs auto scale based on the number of requests passed to the API to handle any increase in requests up to the total TPS limit set for a particular resource based on the pricing tier chosen.

But, to mitigate throttling during autoscaling, the following approaches are recommended.

Implement retry logic in your application.

If you find that you’re being throttled on the number of POST requests, consider adding a delay between the requests.

Increase the workload gradually. Avoid sharp changes.

Create a support request to increase transactions per second(TPS) limit.

I think you might have already requested an increase in the limit so adding a retry logic and using the async analyze API will provide an opportunity to GET the result at a later time. I hope this helps!!

Your answer

romungi-MSFT 48,911 Reputation points Microsoft Employee Moderator

2023-06-15T11:44:08.0333333+00:00

@ryota.matsuda All cognitive services APIs auto scale based on the number of requests passed to the API to handle any increase in requests up to the total TPS limit set for a particular resource based on the pricing tier chosen.

But, to mitigate throttling during autoscaling, the following approaches are recommended.

Implement retry logic in your application.

If you find that you’re being throttled on the number of POST requests, consider adding a delay between the requests.

Increase the workload gradually. Avoid sharp changes.

Create a support request to increase transactions per second(TPS) limit.

I think you might have already requested an increase in the limit so adding a retry logic and using the async analyze API will provide an opportunity to GET the result at a later time. I hope this helps!!

Share via

Server Error Countermeasures for FR

Your answer