Increase MAx_response tokens

Question

Increase MAx_response tokens

Stole Kitanovikj 0

Hello,

Does anyone succeed to increase Max_Response tokens on 4o? I am constantly getting up to 4k tokens but i need 20k or more.

Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-09-27T04:41:12.71+00:00

Hi Stole Kitanovikj,

Thank you for reaching out to the Microsoft Q&A forum.

I understand that you are facing issue with increasing the token limit in gpt-4o.

According to the Azure OpenAI Service quotas and limits, the GPT-4o model has a default maximum token limit of 4096 tokens.
However, we can increase our token limit can be increased based on the region you are operating in.

For example: In the eastus region, gpt-4o supports up to 1 million tokens in one entry and 2 million tokens in another.
Similarly, in the eastus2 region, the limits are also set to 1 million tokens and 2 million tokens.

Here the Screenshot
To help you further, here are some useful references
1.Increase token limits: You can review how to increase your quota in this Azure guide.
2.Check current limits: You can verify the token limits for different regions using this quota and limits guide.

I hope this information helps. Please do let us know if you have any further queries.
Thank you.
Stole Kitanovikj 0 Reputation points

2024-09-27T07:54:08.57+00:00

We would like to point that the we are talking about "number of tokens per model RESPONSE" not request. With this our responses are cut and we do not get the whole response. We need this to be increased but how?
Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-09-27T09:59:50.7533333+00:00

Hi Stole Kitanovikj,

Sorry for the misunderstanding.

For the GPT-4o version (2024-05-13), the maximum response limit is 4096, and this cannot be increased.

However, with the GPT-4o version (2024-08-06), the maximum response limit can be increased up to 16,384 tokens.

Please switch to the GPT-4o (2024-08-06) version to utilize the increased token limit.

Here is the screenshot showing the configuration for deploying the GPT-4 (2024-08-06) :
Here the screenshot of model with the extended max response limit
I hope this information helps.
Thank you
Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-09-30T09:29:39.36+00:00

Hi Stole Kitanovikj,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-10-01T08:02:49.2833333+00:00

Hi Stole Kitanovikj,
Following up to see if the given response was helpful. If you need any issues, please let us know. Thank you!

Your answer

Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-09-27T04:41:12.71+00:00

Hi Stole Kitanovikj,

Thank you for reaching out to the Microsoft Q&A forum.

I understand that you are facing issue with increasing the token limit in gpt-4o.

According to the Azure OpenAI Service quotas and limits, the GPT-4o model has a default maximum token limit of 4096 tokens.
However, we can increase our token limit can be increased based on the region you are operating in.

For example: In the eastus region, gpt-4o supports up to 1 million tokens in one entry and 2 million tokens in another.
Similarly, in the eastus2 region, the limits are also set to 1 million tokens and 2 million tokens.

Here the Screenshot
To help you further, here are some useful references
1.Increase token limits: You can review how to increase your quota in this Azure guide.
2.Check current limits: You can verify the token limits for different regions using this quota and limits guide.

I hope this information helps. Please do let us know if you have any further queries.
Thank you.
Stole Kitanovikj 0 Reputation points

2024-09-27T07:54:08.57+00:00

We would like to point that the we are talking about "number of tokens per model RESPONSE" not request. With this our responses are cut and we do not get the whole response. We need this to be increased but how?
Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-09-27T09:59:50.7533333+00:00

Hi Stole Kitanovikj,

Sorry for the misunderstanding.

For the GPT-4o version (2024-05-13), the maximum response limit is 4096, and this cannot be increased.

However, with the GPT-4o version (2024-08-06), the maximum response limit can be increased up to 16,384 tokens.

Please switch to the GPT-4o (2024-08-06) version to utilize the increased token limit.

Here is the screenshot showing the configuration for deploying the GPT-4 (2024-08-06) :
Here the screenshot of model with the extended max response limit
I hope this information helps.
Thank you
Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-09-30T09:29:39.36+00:00

Hi Stole Kitanovikj,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet.
Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-10-01T08:02:49.2833333+00:00

Hi Stole Kitanovikj,
Following up to see if the given response was helpful. If you need any issues, please let us know. Thank you!

Share via

Increase MAx_response tokens

Your answer