Configuring prompts

When creating a prompt, you can adjust parameters that control how the prompt behaves. In Semantic Kernel, these parameters both control how a function is run by an AI model and how it used by function calling and planners.

For example, you could add settings to the chat prompt from the previous article with the following code

In C#, you can define the following properties of a prompt:

  • Name - the name of the prompt
  • Description - a description of what the prompt does
  • Template format - the format of the prompt template (e.g., semantic-kernel, Handlebars)
  • Input variables - the variables that are used inside of the prompt (e.g., request)
  • Execution settings - the settings for different models that can be used to execute the prompt
// Create a template for chat with settings
var chat = kernel.CreateFunctionFromPrompt(
    new PromptTemplateConfig()
    {
        Name = "Chat",
        Description = "Chat with the assistant.",
        Template = @"{{ConversationSummaryPlugin.SummarizeConversation $history}}
                    User: {{$request}}
                    Assistant: ",
        TemplateFormat = "semantic-kernel",
        InputVariables = new List<InputVariable>()
        {
            new() { Name = "history", Description = "The history of the conversation.", IsRequired = false, Default = "" },
            new() { Name = "request", Description = "The user's request.", IsRequired = true }
        },
        ExecutionSettings =
        {
            {
                "default",
                new OpenAIPromptExecutionSettings()
                {
                    MaxTokens = 1000,
                    Temperature = 0
                }
            },
            {
                "gpt-3.5-turbo", new OpenAIPromptExecutionSettings()
                {
                    ModelId = "gpt-3.5-turbo-0613",
                    MaxTokens = 4000,
                    Temperature = 0.2
                }
            },
            {
                "gpt-4",
                new OpenAIPromptExecutionSettings()
                {
                    ModelId = "gpt-4-1106-preview",
                    MaxTokens = 8000,
                    Temperature = 0.3
                }
            }
        }
    }
);

Parameters used by planner

The description field input_variables array are leveraged by planners to determine how to use a function. The description tells planner what the function does, and the input_variables tells planner how to populate the input parameters.

Because these parameters impact the behavior of planner, we recommend running tests on the values you provide to ensure they are used by planner correctly.

When writing description and input_variables, we recommend using the following guidelines:

  • The description fields should be short and concise so that it does not consume too many tokens when used in planner prompt (but not so short that it is not descriptive enough).
  • Consider the descriptions of other functions in the same plugin to ensure that they are sufficiently unique. If they are not, planner may not be able to distinguish between them.
  • If you have trouble getting planner to use a function, try adding recommendations or examples for when to use the function.

Execution settings used by AI models

In addition to providing parameters for planner, the execution settings also allows you to control how a function is run by an AI model. The following table describes the many of the commonly available settings for models:

Completion Parameter Type Required? Default Description
max_tokens integer Optional 16 The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens can't exceed the model's context length. Most models have a context length of 2048 tokens (except davinci-codex, which supports 4096).
temperature number Optional 1 What sampling temperature to use. Higher values means the model will take more risks. Try 0.9 for more creative applications, and 0 (argmax sampling) for ones with a well-defined answer. We generally recommend altering this or top_p but not both.
top_p number Optional 1 An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both.
presence_penalty number Optional 0 Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
frequency_penalty number Optional 0 Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.

To learn more about the various parameters available for OpenAI and Azure OpenAI models, visit the Azure OpenAI reference article.

Default setting for OpenAI and Azure OpenAI

If you do not provide completion parameters, Semantic Kernel will use the default parameters for the OpenAI API. Learn more about the current defaults by reading the Azure OpenAI API reference article.

Take the next step