Share via

Azure OpenAI Seed parameter

김봉기 0 Reputation points
2024-06-11T00:41:08.9633333+00:00

I am using the AzureOpenAI module and the client.chat.completions.create() function to generate responses. My goal is to receive the same answer each time I ask the same question, ensuring replicable results. I noticed there's a 'seed' parameter that might serve this purpose. However, after testing, the responses still vary with each query. Could you provide guidance on how to achieve consistent, replicable answers using AzureOpenAI?

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,646 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Gowtham CP 3,730 Reputation points
    2024-06-11T04:02:55.3266667+00:00

    Hello 김봉기 ,

    Thank you for reaching out on the Microsoft Q&A .

    To achieve consistent responses with Azure OpenAI's seed parameter, consider the following:

    • Set an integer value for the seed parameter in your client.chat.completions.create() call (e.g., seed=42).
    • Lower the max_tokens value if possible, as higher settings can lead to more variability.
    • Check the system_fingerprint parameter in responses; if this changes with the same seed, there may be backend changes affecting consistency.
    • For consistency, think about pre-computing common responses or using custom logic.

    For more details, refer to Azure's official documentation.

    If the information is useful, please accept the answer and upvote it to assist other community members.

    0 comments
  2. AshokPeddakotla-MSFT 30,511 Reputation points
    2024-06-11T04:27:37.1866667+00:00

    김봉기 Greetings & Welcome to Microsoft Q&A forum!

    I am using the AzureOpenAI module and the client.chat.completions.create() function to generate responses. My goal is to receive the same answer each time I ask the same question, ensuring replicable results. I noticed there's a 'seed' parameter that might serve this purpose. However, after testing, the responses still vary with each query. Could you provide guidance on how to achieve consistent, replicable answers using AzureOpenAI?

    Did you check these docs Learn how to use reproducible output (preview) and ChatCompletionsOptions.Seed Property already?

    Reproducible output is only currently supported with the following models.

    API Version : Support for reproducible output was first added in API version 2023-12-01-preview

    When you want to use reproducible outputs, you need to set the seed to the same integer across chat completions calls. You should also match any other parameters like temperature, max_tokens, etc.

    Do let me know if that helps.

    0 comments