How to implement real time stream response using Azure Functions .NET?

Question

I found a topic which describes my needs. The provided answer is great looking solution that I'm looking for on C#. Is there any possibility to do the same as provided answer for python but on C#?

import json
import logging
import time

import azure.functions as func

def main(req: func.HttpRequest) -> func.HttpResponse:
    logging.info('Python HTTP trigger function processed a request.')
    req_body = req.get_body()
    req_json = json.loads(req_body)
    question = req_json['question']

    prompt = """
    # my prompt here
    """

    # query_engine is the engine to send query to Azure OpenAI with reference to the file text indexing
    llamaIndexresponse = query_engine.query((prompt+question))

    # Set the content type to text/event-stream
    headers = {
        'Content-Type': 'text/event-stream',
        'Cache-Control': 'no-cache',
        'Connection': 'keep-alive'
    }

    # Send the headers to the client
    response = func.HttpResponse(headers=headers)

    # Send the response to the client in real-time using SSE
    response.streaming = True

    # Send the initial SSE event to the client
    response.write('event: message\n')
    response.write('data: {}\n\n'.format(json.dumps({'response': 'Processing...'})))

    # Send the SSE events to the client
    while not llamaIndexresponse.done:
        response.write('event: message\n')
        response.write('data: {}\n\n'.format(json.dumps({'response': str(llamaIndexresponse.response).strip()})))
        time.sleep(1)

    # Send the final SSE event to the client
    response.write('event: message\n')
    response.write('data: {}\n\n'.format(json.dumps({'response': str(llamaIndexresponse.response).strip()})))

    return response

Answer 1

Hello @Andrii K - Thanks for reaching out, and engaging us on the MS Q&A community. Platform-wise, the HTTP feature in Azure Functions is fronted with Azure Load Balancer which imposes 230 Timeout. This will interrupt the persistent connection that SSE needs to update the Client. As such, Azure Functions may not be the best choice for your scenario for now. Sorry to be the bearer of bad news here but I'll update you here if I find out anything new.

---

Please "Accept Answer" if the answer is helpful so that others in the community may benefit from your experience.