Error uploading JSONL files for OpenAI Custom Model

Question

Error uploading JSONL files for OpenAI Custom Model

Tan Luong 5

I followed this tutorial: https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/fine-tuning?pivots=programming-language-python

I was able to upload training and testing data on East US but got internalFailure on South Central US. I also used OpenAI CLI data preparation tool and uploaded via Azure OpenAI Studio but still didn't work.

Please advise. Thanks!

romungi-MSFT 48,786 Reputation points Microsoft Employee

2023-03-30T12:05:23.3+00:00

@Tan Luong Is it possible to post a snippet of your file or share the file that failed upload? I was working on a similar error from another user and used my resources in EastUS and SouthCentralUS. In both cases the file was successfully uploaded. Please see this thread for details.

https://learn.microsoft.com/en-us/answers/questions/1194475/azure-openai-testing-and-training-internal-error-w

Tan Luong 5

Hi @romungi-MSFT , thanks for response. I just tried but still no luck. My data as below:

{"prompt": "When I go to the store, I want an", "completion": "apple"},
{"prompt": "When I go to work, I want a", "completion": "coffee"},
{"prompt": "When I go home, I want a", "completion": "soda"}

Tan Luong 5

Hi @romungi-MSFT , Thanks for response. I just tried again but no luck. My data as below:

{"prompt": "When I go to the store, I want an", "completion": "apple"},
{"prompt": "When I go to work, I want a", "completion": "coffee"},
{"prompt": "When I go home, I want a", "completion": "soda"}

Tan Luong 5 Reputation points

2023-03-30T18:15:27.81+00:00

Sorry for confusion. My file doesn't have commas. I copied from the tutorial's codes.
I tried your data, it shows different error: "InternalServerError: Gateway cannot authenticate upstream services. Please contact Microsoft for help"
Tan Luong 5 Reputation points

2023-03-30T18:26:01.7466667+00:00

It works now. I didn't change anything. It could be server problem.

Anyway, thank you so much for your support. I really appreciate it.
romungi-MSFT 48,786 Reputation points Microsoft Employee

2023-03-31T11:16:09.89+00:00

That's great Tan Luong. Service team for Azure OpenAI seems to have deployed some hotfixes for the internal failure issue. Your resource could have been one of them that got impacted. Thanks!!

1 answer

Your answer

romungi-MSFT 48,786 Reputation points Microsoft Employee

2023-03-30T12:05:23.3+00:00

@Tan Luong Is it possible to post a snippet of your file or share the file that failed upload? I was working on a similar error from another user and used my resources in EastUS and SouthCentralUS. In both cases the file was successfully uploaded. Please see this thread for details.

https://learn.microsoft.com/en-us/answers/questions/1194475/azure-openai-testing-and-training-internal-error-w
Tan Luong 5 Reputation points

2023-03-30T16:44:02.9666667+00:00

Hi @romungi-MSFT , thanks for response. I just tried but still no luck. My data as below:

{"prompt": "When I go to the store, I want an", "completion": "apple"}, {"prompt": "When I go to work, I want a", "completion": "coffee"}, {"prompt": "When I go home, I want a", "completion": "soda"}
Tan Luong 5 Reputation points

2023-03-30T16:46:03.7966667+00:00

Hi @romungi-MSFT , Thanks for response. I just tried again but no luck. My data as below:

{"prompt": "When I go to the store, I want an", "completion": "apple"}, {"prompt": "When I go to work, I want a", "completion": "coffee"}, {"prompt": "When I go home, I want a", "completion": "soda"}
Tan Luong 5 Reputation points

2023-03-30T18:15:27.81+00:00

Sorry for confusion. My file doesn't have commas. I copied from the tutorial's codes.
I tried your data, it shows different error: "InternalServerError: Gateway cannot authenticate upstream services. Please contact Microsoft for help"
Tan Luong 5 Reputation points

2023-03-30T18:26:01.7466667+00:00

It works now. I didn't change anything. It could be server problem.

Anyway, thank you so much for your support. I really appreciate it.
romungi-MSFT 48,786 Reputation points Microsoft Employee

2023-03-31T11:16:09.89+00:00

That's great Tan Luong. Service team for Azure OpenAI seems to have deployed some hotfixes for the internal failure issue. Your resource could have been one of them that got impacted. Thanks!!

Answer 1

Tan Luong I think in the above case when I added the above text to my file, I noticed the commas are added. I think these are not required for JSONL. I removed them and ran the file through the data validation tool which them updated my file with the below suggestions.

User's image

Following these guidelines, changed the file to following:

{"prompt":"When I go to the store test, I want an ->","completion":" apple."}
{"prompt":"When I go to work, I want a ->","completion":" coffee."}
{"prompt":"When I go home, I want a ->","completion":" soda."}
{"prompt":"Item is a handbag. Colour is army green. Price is midrange. Size is small.->","completion":"This stylish small green handbag will add a unique touch to your look, without costing you a fortune."}
{"prompt": "When I go to the store, I want an", "completion": "apple"}
{"prompt": "When I go to work, I want a", "completion": "coffee"}
{"prompt": "When I go home, I want a", "completion": "soda"}

Some of the prompts and completions are from my previous file.

The above content uploaded as a file was successfully uploaded.

User's image

I hope this helps!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

Error uploading JSONL files for OpenAI Custom Model

1 answer

Your answer