Reducing randomness of responses in turbo model

Question

How can I reduce the randomness of responses to the same prompt and configuration in my turbo model? I have already set temperature=0 and top_p=0, but I would appreciate additional help with the settings.

Answer 1

Thanks for the question, It now works in AOAI through SDK with the latest v1106 models:https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/reproducible-output

New seed indeed helps to reproduce the same output (although with some occasional differences).

Generating reproducible output in GPT-4-Turbo v1106 with the "seed" parameter in Python SDK v1.x

Here is the repo for the same.

Answer 2

Hi Divya67

Thanks for the question, If you've already set temperature=0 and top_p=0 it means you have already tried this and doing this is a good way to achieve this I'll suggest an alternative and hope it helps.

Set a seed for the random number generator. This make sure that the same prompt and configuration consistently produce the same output. , seeding the random number generator provides a level of determinism and may improve randomness.

import numpy as np

import tensorflow as tf

tf.random.set_seed(your_seed)

np.random.seed(your_seed)

you can Instead of relying on the online API, consider using the model checkpoints directly. This way, you can have more control over the inference process and ensure reproducibility. for more info stay connected with the official documentation

Azure OpenAI Service Documentation

If you find this answer useful, kindly accept thanks much.