How to get streaming response for chat completion using AsyncAzureOpenAI with stream=True or any other way

Question

I am trying to get streaming response for chat completion using AsyncAzureOpenAI with stream=True, but I'm getting a null object output. I am using the following code:

import os
import openai
import asyncio
from openai import AzureOpenAI, AsyncAzureOpenAI

import os
import asyncio
from openai import AsyncAzureOpenAI

azure_openai_client = AsyncAzureOpenAI(
azure_endpoint = "",
api_key="some-key",
api_version="2023-07-01-preview"
)

async def get_response(message):
response = await azure_openai_client.chat.completions.create(
model = 'GPT35',
temperature = 0.4,
messages = [
{"role": "user", "content": message}
],
stream=True
)

async for chunk in response:
if "choices" in chunk and len(chunk["choices"]) > 0:
  print(chunk.choices[0].message.content, end="") # chunk.choices[0].delta.content - this also not working
asyncio.run(get_response('What is chatgpt?'))

And I'm getting the following error output:

Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001D871705160>
Traceback (most recent call last):
  File "C:\Program Files\Python38\lib\asyncio\proactor_events.py", line 116, in __del__  
  File "C:\Program Files\Python38\lib\asyncio\proactor_events.py", line 108, in close    
  File "C:\Program Files\Python38\lib\asyncio\base_events.py", line 719, in call_soon    
  File "C:\Program Files\Python38\lib\asyncio\base_events.py", line 508, in _check_closed
RuntimeError: Event loop is closed

Can someone show me an example on how to get streaming response for chat completion in openai version 1.x (1.3.9 or any latest version)?

Note:

await openai.ChatCompletion.acreate()

This used to work in version 0.x, but I'm now migrating to 1.x version and stream response is not working.

Answer 1

@Anuj Agarwal I tried the below sample and it worked with my Azure OpenAI deployment with streaming enabled. Please see the screen shot below.

import asyncio


# gets API Key from environment variable OPENAI_API_KEY
client = AsyncAzureOpenAI(azure_endpoint = "your_endpoint",api_key="your_key",api_version="2023-09-01-preview")

async def main() -> None:
    stream = await client.chat.completions.create(
        model="your_deployment_name",
        messages = [ {"role": "user", "content": "What is chatgpt?"} ],
        stream=True,
    )

    #async for data in stream:
    #    print(data.model_dump_json())
    #    print("test")

    async for choices in stream:
        print(choices.model_dump_json(indent=2))    
        print()

asyncio.run(main())

Referencing the same issue from SDK repo.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.