Logging of prompts for admin review on Azure OpenAI Service

Dh1 25 Reputation points

Hello everyone,

I have deployed internally an Azure OpenAI endpoint. I however need to know what my users sent to it for compliance reasons.

It does not seem to be the case that the product has a built-in feature to record those prompts and allow admins to read them afterwards.

I could do it myself by setting a proxy server, and users would first send prompts to it, logging prompts here, before sending the prompts to my internal Azure OpenAI endpoint, but it would be extra work.

Do you think this feature of prompt logging is coming any time soon?



Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
1,431 questions
0 comments No comments
{count} vote

Accepted answer
  1. Pramod Valavala 19,356 Reputation points Microsoft Employee

    @Dh1 While a lot of data is made available for monitoring purposes, the actual payloads for requests/responses are not persisted by default.

    You would need to implement your own service which can perform this by intercepting requests/responses as you've mentioned and storing them appropriately.

    You can refer to the sample web app used for OpenAI on your data as reference, where conversations are persisted into CosmosDB.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful