I've noticed that the knowdage of the model, gpt-4-turbo, varies between Azure OpenAI and OpenAI.

Question

I've noticed that the knowdage of the model, gpt-4-turbo, varies between Azure OpenAI and OpenAI.

David Swift 105

I was thrilled to discover recently that Azure OpenAI has launched GPT-4 Turbo, boasting a 128k context window and a maximum 4k token output. Initially, I assumed Azure OpenAI's GPT-4 Turbo was identical to GPT-4-1106-preview, only to find it was not so. To clarify, Azure OpenAI's GPT-4-1106-preview is a model featuring an 8k context window and also a maximum 4k output. I'm eager to understand how I can access GPT-4 Turbo. Given our past disappointment with OpenAI's performance, we're keen on utilizing this sophisticated model in Azure OpenAI. I'd greatly appreciate any guidance provided.

David Swift 105 Reputation points

2023-11-21T14:43:22.6333333+00:00

I found that there is a mistake; it is 8k context in the playground but it can really be given a 100k input via API. However, the problem now is that the knowledge doesn't seem the same between the OpenAI model and the Azure OpenAI model.
David Swift 105 Reputation points

2023-11-21T14:45:05.15+00:00

I'm truly eager to find the solution; it's quite baffling.
Saurabh Sharma 21,101 Reputation points Microsoft Employee

2023-11-21T21:05:22.75+00:00

Hi @David Swift

Welcome to Microsoft Q&A! Thanks for posting the question.

Azure OpenAI's GPT-4 (1106-preview) model is GPT-4 Turbo only with 128K context window.

Please refer to the documentations - Regional quota limits and GPT-4 and GPT-4 Turbo Preview models.

Can you please provide the screenshot where you are seeing 8k context window on the playground.

Do you mean the following information that appears on Azure Open AI Studio?

Thanks

Saurabh
David Swift 105 Reputation points

2023-11-22T02:19:12.63+00:00

Yes, that's where I found it only has an 8k context window, but when I use it through the API, I find it is a real 128k model. It is a small mistake on your webpage, I think. I am now curious about the knowledge it has. It seems between you and OpenAI, let me show you. Just ask a question about something going on lately, for example, ask it Based on your understanding, when was Apple's most recent event and what products were released?

here is the answer of azure ai

while openai

I am truly keen on discovering the solution, the issue of knowledge.
Liam Andrew 5 Reputation points

2023-11-22T09:18:53.96+00:00

The gpt-4-1106-preview (at least the one available in the Sweden Central region resource) is not GPT-4 Turbo, it has a knowledge cutoff date of September 2021.
Simone Gallo 0 Reputation points

2023-11-22T09:51:41.8633333+00:00

Same issue here.

Reading latest comments here: https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/azure-openai-service-launches-gpt-4-turbo-and-gpt-3-5-turbo-1106/ba-p/3985962 seems like the model has "bugged" responses but knowledge update to 2023.

From my tests, it has no knowledge even for 2022. Moreover, tokens are not counted in the metrics monitoring dashboard
lisa.cao 5 Reputation points

2023-11-22T10:10:49.4766667+00:00

I've seen many users provide feedback on this issue. When does Azure plan to provide a solution?

David Swift 105 Reputation points

2023-11-21T14:43:22.6333333+00:00

I found that there is a mistake; it is 8k context in the playground but it can really be given a 100k input via API. However, the problem now is that the knowledge doesn't seem the same between the OpenAI model and the Azure OpenAI model.
David Swift 105 Reputation points

2023-11-21T14:45:05.15+00:00

I'm truly eager to find the solution; it's quite baffling.
Saurabh Sharma 21,101 Reputation points Microsoft Employee

2023-11-21T21:05:22.75+00:00

Hi @David Swift

Welcome to Microsoft Q&A! Thanks for posting the question.

Azure OpenAI's GPT-4 (1106-preview) model is GPT-4 Turbo only with 128K context window.

Please refer to the documentations - Regional quota limits and GPT-4 and GPT-4 Turbo Preview models.

Can you please provide the screenshot where you are seeing 8k context window on the playground.

Do you mean the following information that appears on Azure Open AI Studio?

Thanks

Saurabh
David Swift 105 Reputation points

2023-11-22T02:19:12.63+00:00

Yes, that's where I found it only has an 8k context window, but when I use it through the API, I find it is a real 128k model. It is a small mistake on your webpage, I think. I am now curious about the knowledge it has. It seems between you and OpenAI, let me show you. Just ask a question about something going on lately, for example, ask it Based on your understanding, when was Apple's most recent event and what products were released?

here is the answer of azure ai

while openai

I am truly keen on discovering the solution, the issue of knowledge.
Liam Andrew 5 Reputation points

2023-11-22T09:18:53.96+00:00

The gpt-4-1106-preview (at least the one available in the Sweden Central region resource) is not GPT-4 Turbo, it has a knowledge cutoff date of September 2021.
Simone Gallo 0 Reputation points

2023-11-22T09:51:41.8633333+00:00

Same issue here.

Reading latest comments here: https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/azure-openai-service-launches-gpt-4-turbo-and-gpt-3-5-turbo-1106/ba-p/3985962 seems like the model has "bugged" responses but knowledge update to 2023.

From my tests, it has no knowledge even for 2022. Moreover, tokens are not counted in the metrics monitoring dashboard
lisa.cao 5 Reputation points

2023-11-22T10:10:49.4766667+00:00

I've seen many users provide feedback on this issue. When does Azure plan to provide a solution?

Answer 1

Saurabh Sharma 21,101 Microsoft Employee

@David Swift

For your first issue about the context window, the product team has confirmed that this is a UI limitation. I am still waiting for their confirmation on whether they will update the studio UX to allow a higher input token limit by default.

Now, regarding your second issue of training, this is an expected behavior and part of the problem is that it both doesn't know about itself (refer to - When I ask GPT-4 what model it's running it tells me it's running GPT-3. Why does this happen?) and without RAG augmentation isn't a 100% reliable source of facts in general though it will often get things right if it has seen enough instances of something in its training data.

For newer info it isn't surprising that older info that would have more occurrences in the training data might crowd out newer info depending on how a question is asked. The training data definitely goes beyond 2021.

Here is an example question you can ask to see that training data goes beyond 2021:

User's image

Product team is internally working to get this information updated to under FAQs as well.

Please let me know if you have any questions.

Thanks

Saurabh

Please 'Accept as answer' and Upvote if it helped so that it can help others in the community looking for help on similar topics.

David Swift 105 Reputation points

2023-11-23T01:20:48.6466667+00:00

Thank you so much for your response; it's immensely helpful. You've highlighted the mechanics of a language model; it processes textual input to determine the most likely reply. Surely, OpenAI has modified the model in some way, perhaps merely with a prompt, who can tell. Nevertheless, I gave prompts a shot in azure openai model, and indeed they work!

I appreciate your reminder that this text model doesn't possess human qualities; its output is influenced by the input provided. The training data it's based on serves as one of the potential factors impacting its results.
David Swift 105 Reputation points

2023-11-23T01:32:48.1+00:00

So, could Azure provide the solution we need, without requiring us to seek improved outcomes via a prompt or such?
Saurabh Sharma 21,101 Reputation points Microsoft Employee

2023-11-23T01:39:34.9066667+00:00

David Swift Thank you for your input. I don't have a definitive answer for you at the moment, but I will share your feedback with the product team.

Thanks

Saurabh
Simone Gallo 0 Reputation points

2023-11-23T09:12:13.3033333+00:00

@David Swift could you please reach me at my email (simo.gallo2@hotmail.it)? I would very much like to discuss these issues (that I'm facing too) with you.
David Swift 105 Reputation points

2023-11-23T11:03:02.14+00:00

Yes, of course, I have emailed you.

I've noticed that the knowdage of the model, gpt-4-turbo, varies between Azure OpenAI and OpenAI.

0 additional answers

Activity