Is there a way to stream the output from the Azure OpenAI API so that UI does not have to wait for the entire response? Thanks, Ravi.

@Ravi K Both the completions and chat APIs support a stream parameter, which when set to true , streams the response back from the model via Server Sevent Events (SSE) . The same is supported in the SDKs available as well. For example, in C#, you can use the GetChatCompletionsStreaming / GetChatCompletionsStreamingAsync methods for chat completions and the GetCompletionsStreaming / GetCompletionsStreamingAsync methods for regular completions.

Streaming with Azure OpenAI API

Accepted answer

Pramod Valavala 20,636 Reputation points Microsoft Employee

2023-10-30T20:11:11.67+00:00

@Ravi K Both the completions and chat APIs support a stream parameter, which when set to true, streams the response back from the model via Server Sevent Events (SSE).

The same is supported in the SDKs available as well. For example, in C#, you can use the GetChatCompletionsStreaming/GetChatCompletionsStreamingAsync methods for chat completions and the GetCompletionsStreaming/GetCompletionsStreamingAsync methods for regular completions.
Please sign in to rate this answer.
Ravi K 70 Reputation points

2023-10-31T01:49:25.4166667+00:00

Thank you.

Anonymous

2023-11-21T09:59:29.73+00:00

Is it possible to do that using Node? Are there any examples of this?

Pramod Valavala 20,636 Reputation points Microsoft Employee

2023-11-21T14:18:20.45+00:00

@Daniel Georgiev Yes. You can use the listChatCompletions/listCompletions methods can be used and here is a sample for the same.

Christopher Quenneville 0 Reputation points

2024-01-04T21:46:02.0733333+00:00

The code sample provided by azure says the following:

#Note: This code sample requires OpenAI Python library version 0.28.1 or lower.

I was under the impression that v0.28.1 doesn't support streaming.

Is the code sample obsolete? Can I use OpenAI's newer versions?

Pramod Valavala 20,636 Reputation points Microsoft Employee

2024-01-05T13:47:02.7066667+00:00

@Christopher Quenneville For Python, the recommendation is to use the openai package itself, so the same samples in the README should be held.

If you are facing an error, it would be best to create a new question to get help for it.

Diallo, Francis Patrick 0 Reputation points

2024-04-22T06:29:18.4666667+00:00

Is it possible to do that using Java? Are there any examples of this? Thank you!
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Streaming with Azure OpenAI API

0 additional answers

Your answer