Server Sent Events not working in App Service (Linux, containers)

Alejandro Martín 5 Puntos de reputación
2023-11-22T10:14:03.7+00:00

We have an app running with custom containers using docker compose in Azure App Service (Linux plan). One of the containers expose an HTTP API endpoint that make call to Azure OpenAI Service and stream the response (text/event-stream mime-type). So the workflow is:

  • Browser make a request to our API.
  • API craft a request to Azure OpenAI.
  • API streams Azure OpenAI back to browser using Server Sent Events.

It's working fine using the same docker compose and containers on local machines. But when we deploy to App Service it doesn't work as expected, it doesn't start downloading the stream until the response is fully completed. Streaming from Azure OpenAI to our container works fine, but not from our container to user's browser when deployed on App Service, we believe that there's an internal proxy buffering the streamed data on App Service side.

Expected behaviour, reasonable delay until getting the first bytes from the streamed response:

response_local

Same request when deployed in Azure App Service:Screenshot from 2023-11-22 10-51-46

As you can see the response is not streamed until completed.

Is this a known issue? Is there a way to fix this behaviour using App Service configuration?

Kind regards.

Sin supervisión
Sin supervisión
Etiqueta no supervisada por Microsoft.
198 preguntas
0 comentarios No hay comentarios
{count} voto