Minimum PTU Requirements for GPT-4o in US Regions

Question

Hi experts,

What is the minimum PTU requirement for deploying GPT-4o in any US region?

We are estimating roughly 1000 transactions per minute, with approximately 100k tokens in input and output. Traditional quota requests have not alleviated our concerns about potential production failures due to rate limitations, despite having a retry and queue mechanism in place.

We have requested PTUs but learned that a minimum PTU requirement exists for them to be reflected in the deployment type. Has anyone else faced challenges deploying GPT-4o for production workloads? What is the process, and how can it be navigated?

Answer 1

Hi Rohit Shetty

Welcome to Microsoft Q&A Forum, thank you for posting your query here!

Given your estimate of 1000 transactions per minute with approximately 100k tokens in input and output, you will need to ensure that your deployment meets these minimum PTU requirements to avoid potential production failures. If you have already requested PTUs, you should confirm that your request aligns with these minimums for the deployment to be effective.

The minimum PTU requirement for deploying GPT-4o in US regions is as follows:

• Global & data zone provisioned minimum deployment: 15 PTUs
• Regional provisioned minimum deployment: 50 PTUs

If you encounter challenges during deployment, it may be beneficial to consult the Azure OpenAI Capacity calculator to better understand your throughput needs and ensure that your capacity aligns with your expected workload.

Please leverage built in capacity calculator from Sweden Central region to estimate your PTU requirement as mentioned in this page.

Kindly refer below link: Provisioned Throughput Units (PTU)

provisioned throughput

Thank You.