error with the playground chat when I try to add my own data

Question

error with the playground chat when I try to add my own data

John Lipinski 70

I am using the Chat in the playground area of Azure AI Studio. When I try to use the chat together with my own data, I received the following error in the window:
"LLM call failed for chat completion endpoint. Please set OAIGenerationConfig(is_chat=True)"

I am using the gpt35 turbo model. This error only occurs when I add my own data. If I remove my data, the chat works again. This error when attempting to use my own data occurs regardless of whether I add my own data on the fly in the wizard or if I use an index previously successfully created in Cognitive Search.
Is there something wrong with my configuration or is there some limit behind the scenes that I cannot see?

Stephen Yeadon 0 Reputation points

2023-07-04T12:09:55.47+00:00

I have exactly the same problem. For my data I am using the Cognitive search hotels-sample. Where would we be expected to set OAIGenerationConfig(is_chat=True)
Prichard,Derek,NORTH YORK,IT Canada 0 Reputation points

2023-07-06T14:53:27.8633333+00:00

I am experiencing the exact same issue. Bring your own data with the error that I need to set "OAIGenerationConfig(is_chat=True)"
Marco G 16 Reputation points

2023-08-01T12:26:32.02+00:00

According to https://github.com/microsoft/sample-app-aoai-chatGPT/issues/114#issuecomment-1659638966
This error may happen if Azure OpenAI service in your zone is not healty.
You can check service health under Resource Health at the bottom of left menu.
At the moment, in West Europe zone, most is experiencing the same error message.
Hope this helps, regards.

Accepted answer

7 additional answers

Your answer

Stephen Yeadon 0 Reputation points

2023-07-04T12:09:55.47+00:00

I have exactly the same problem. For my data I am using the Cognitive search hotels-sample. Where would we be expected to set OAIGenerationConfig(is_chat=True)
Prichard,Derek,NORTH YORK,IT Canada 0 Reputation points

2023-07-06T14:53:27.8633333+00:00

I am experiencing the exact same issue. Bring your own data with the error that I need to set "OAIGenerationConfig(is_chat=True)"
Marco G 16 Reputation points

2023-08-01T12:26:32.02+00:00

According to https://github.com/microsoft/sample-app-aoai-chatGPT/issues/114#issuecomment-1659638966
This error may happen if Azure OpenAI service in your zone is not healty.
You can check service health under Resource Health at the bottom of left menu.
At the moment, in West Europe zone, most is experiencing the same error message.
Hope this helps, regards.

Answer 1

Luis Barcenas 95

I FOUND THE ANSWER!

The problem is with the DEPLOYMENT NAME.

If your deployment name includes "_" then this issue will present.

According to MSFT a fix will be roll out next week that should solve this.

Drop the "_" from your deployment name and this should work.

Good luck!

John Lipinski 70 Reputation points

2023-07-10T17:27:07.75+00:00

Wow! Thank you! I will give that a shot later today and report back. Thanks for the tip and the follow up.
Tom Reeman 5 Reputation points

2023-07-11T14:16:58.0533333+00:00

Thanks so much! This fixed my issue
Joan Navas 5 Reputation points

2023-07-11T16:03:14.4166667+00:00

Fixed here too.

Thanks!!
John Lipinski 70 Reputation points

2023-07-11T18:57:33.06+00:00

Worked like a charm. Just created another deployment of the 35 turbo model with no hyphen in the name and it worked perfectly. Thank you so much for taking the time to respond. Much appreciated!
Marco G 16 Reputation points

2023-07-28T07:03:04.9566667+00:00

mmm... i am in west europe and have not underscore in my deployment name... not sure if that is the problem.

Error

Server disconnected
LLM call failed for chat completion endpoint. Please set OAIGenerationConfig(is_chat=True)
Milosz Lopatto 0 Reputation points

2023-07-28T16:47:13.05+00:00

@Marco G please make sure that there is no hyphen in the name of LLM model deployment in Azure AI Studio -> Deployments. I have just changed "ML-test" to "MLtest" and after this simple change it started working.

Note: I'm in West Europe as well.

UPDATE: I agree, after some time it stops working once again.
Marco G 16 Reputation points

2023-07-31T08:16:16.6433333+00:00

i have not hypens there... anyway i think that every time you change starts working but after some time it stops.

Please check this: https://github.com/microsoft/sample-app-aoai-chatGPT/issues/114
Santhosh Alagirisamy 0 Reputation points

2023-07-31T10:26:24.62+00:00

I removed the underscore & hyphens from my deployment name but still got the same error message.

using GPT 35 Turbo model

Error

Server disconnected
LLM call failed for chat completion endpoint. Please set OAIGenerationConfig(is_chat=True)
Marco G 16 Reputation points

2023-07-31T10:36:36.4733333+00:00

trying it on sunday night (CET) is ok while in working days/hours no: it seems to me a service overload...

Answer 2

Luke Murray 11,436 MVP Volunteer Moderator

Hi, John

I've had similar issues (but not identical), when creating my Azure OpenAI instance via PowerShell - I think it uses an old API instance with the incorrect endpoints. How was this created?

Are you able to try recreating OpenAI using the Portal and trying that?

Luke Murray 11,436 Reputation points MVP Volunteer Moderator

2023-07-05T08:49:45.8533333+00:00

If you need to recreate your index with Cognitive services, a recent blog article of mine, can be referenced: https://luke.geek.nz/azure/Bring-Your-Data-to-Life-with-Azure-OpenAI/
John Lipinski 70 Reputation points

2023-07-05T12:02:47.3366667+00:00

Thanks for the comment. I created the entire instance within the Azure portal.
Marco G 16 Reputation points

2023-07-28T07:07:28.7833333+00:00

I also did it with the portal and i am in west europe and without underscores in deployment.

Error

Server disconnected
LLM call failed for chat completion endpoint. Please set OAIGenerationConfig(is_chat=True)

strange behaviour..

Answer 3

Konstantinos Passadis 19,591 MVP

Hello @John Lipinski !

Welcome to Microsoft QnA!

I tried to recreate , even if i do not have much info so please send us your feedback and more details if possible

Create Storage Account and a Blob
Upload a sample File ( https://github.com/MicrosoftLearning/mslearn-ai900/blob/main/data/qna_bot/margies_faq.docx) - Just to be sure i have not any errors in my data , suggest you to do the same for testing...
Create a Cognitive Search Standard resource , created the Indexer from the Blob Data
Start the chat and get the results

I hope this helps!

Kindly mark the answer as Accepted and Upvote in case it helped!

Regards

Tetera, Jakub 6 Reputation points

2023-07-04T07:38:20.4633333+00:00
Hi, I'm having the exact same issue. This didn't help. The result is the same.

The error contains an URL - https://<resource>.openai.azure.com/openai/deployments/<deployment>/chat/completions?api-version=2023-03-15-preview. This URL results in the following result:

{"error":{"code":"404","message": "Resource not found"}}

I'm wondering why. Everything works perfectly fine until I try to add my own data...
Konstantinos Passadis 19,591 Reputation points MVP

2023-07-04T16:47:43.6433333+00:00

Hello

Please try with the File i posted

I will post the exact steps!

I hope this helps!

Kindly mark the answer as Accepted and Upvote in case it helped!

Regards
Manea, Daniela 20 Reputation points

2023-07-05T08:33:06.4933333+00:00

I get the same error. Can you post the exact steps on how to create the Indexer from the Blob Data. Many thanks

Answer 4

Ziggy Zulueta 495 MVP

hi @John Lipinski , Where is your open AI resource deployed? I get a bunch of weird errors in the EAST US location but when I switch over to another region like Europe everything is ok. Probably because the location has so many users and it lacks the GPU. Can you try deploying again in another region?

Tatkare, Jyoti 0 Reputation points

2023-07-05T10:04:07.8066667+00:00

I also got the same error

my resource group is created in EAST US

Error

400, message='model_error', url=URL('https://xxxxxx/openai/deployments/xxxxx/chat/completions?api-version=2023-03-15-preview')

LLM call failed for chat completion endpoint. Please set OAIGenerationConfig(is_chat=True)
John Lipinski 70 Reputation points

2023-07-05T12:04:06.0333333+00:00

Thanks for taking the time to comment. Mine was deployed in the East US region. I will try to see about options involving other regions.

Answer 5

Konstantinos Passadis 19,591 MVP

Hello all!!

Today later i will post the exact procedure i did in West Europe

In case you want me to try another region please inform me!

Regards

Tatkare, Jyoti 0 Reputation points

2023-07-05T10:21:48.02+00:00

Hi Konstantinos Passadis

Could you please try to create in EAST US region

I can't deploy my model in Europe due to quota limit

Thanks in advance
Chris Witmore 0 Reputation points

2023-07-15T08:44:19.33+00:00

US Central would be helpful to see.

I tried removing the underscore “_” (as the ANSWER suggested) from the deployment name. This didn’t resolve the upload issue for me.

Whether I go through the Azure Blob or Upload your data options, I’m still getting errors and can’t proceed. 1) Can’t find the container in the Azure Blob. Yet the container is in the storage group and has data in it. 2) The upload is prevented with a message saying Azure needs permission to access the resource, however, CORS is greyed out.
Marco G 16 Reputation points

2023-07-31T12:56:34.9933333+00:00

hey @Konstantinos Passadis could you please check if it is still working or getting the errors in the post?

Share via

error with the playground chat when I try to add my own data

7 additional answers

Your answer