Understanding AI plugins in Semantic Kernel

Plugins are the fundamental building blocks of Semantic Kernel and can interoperate with plugins in ChatGPT, Bing, and Microsoft 365. With plugins, you can encapsulate capabilities into a single unit of functionality that can then be run by the kernel. Plugins can consist of both native code and requests to AI services via prompts.

What is a plugin?

To drive alignment across the industry, we've adopted the OpenAI plugin specification as the standard for plugins. This will help create an ecosystem of interoperable plugins that can be used across all of the major AI apps and services like ChatGPT, Bing, and Microsoft 365.

For developers using Semantic Kernel, this means any plugins you build can be exported so they are usable in ChatGPT, Bing, and Microsoft 365. This allows you to increase the reach of your AI capabilities without rewriting code. It also means that plugins built for ChatGPT, Bing, and Microsoft 365 can be imported into Semantic Kernel seamlessly.

To show how to make interoperable plugins, we've created an in-depth walkthrough on how to export a Semantic Kernel plugin as an OpenAI plugin using OpenAI's specification. You can find the walkthrough in the Create and run OpenAI plugins section.

What does a plugin look like?

At a high-level, a plugin is a group of functions that can be exposed to AI apps and services. The functions within plugins can then be orchestrated by an AI application to accomplish user requests. Within Semantic Kernel, you can invoke these functions either manually or automatically with function calling or planners.

Just providing functions, however, is not enough to make a plugin. To power automatic orchestration with a planner, plugins also need to provide details that semantically describe how they behave. Everything from the function's input, output, and side effects need to be described in a way that the AI can understand, otherwise, the planner will provide unexpected results.

For example, in the WriterPlugin plugin, each function has a semantic description that describes what the function does. A planner can then use these descriptions to choose the best functions to call to fulfill a user's ask.

In the picture on the right, a planner would likely use the ShortPoem and StoryGen functions to satisfy the users ask thanks to the provided semantic descriptions.

Adding functions to plugins

Now that you know what a plugin is, let's take a look at how to create one. Within a plugin, you can create two types of functions: prompts and native functions. The following sections describe how to create each type. For further details, please refer to the Creating prompts and Creating native functions sections.

Native functions

With native functions, you can have the kernel call C# or Python code directly so you can manipulate data or perform other operations. In this way, native functions are like the hands of your AI app. They can be used to save data, retrieve data, and perform any other operation that you can do in code that is ill-suited for LLMs (e.g., performing calculations).

Instead of providing a separate configuration file with semantic descriptions, planners are able to use annotations in the code to understand how the function behaves. Below are examples of the annotations used by planner in both C# and Python for out-of-the-box native functions.

C#

The following code is an excerpt from the DocumentSkill plugin, which can be found in the document plugin folder in the GitHub repository. It demonstrates how you can use the SKFunction and SKFunctionInput attributes to describe the function's input and output to planner.

[SKFunction, Description("Read all text from a document")]
public async Task<string> ReadTextAsync(
   [Description("Path to the file to read")] string filePath
)
{
    this._logger.LogInformation("Reading text from {0}", filePath);
    using var stream = await this._fileSystemConnector.GetFileContentStreamAsync(filePath).ConfigureAwait(false);
    return this._documentConnector.ReadText(stream);
}

Java

The following code is an excerpt from the MathPlugin plugin, which can be found in the examples folder in the GitHub repository. It demonstrates how you can use the kernel_function and Annotated decorators to describe the function's input and output to planner.

@DefineKernelFunction(name = "add", description = "Add two numbers")
public static double add(
      @KernelFunctionParameter(name = "number1", description = "The first number to add", type = double.class) double number1,
      @KernelFunctionParameter(name = "number2", description = "The second number to add", type = double.class) double number2) {
   return number1 + number2;
}

Python

The following code is an excerpt from the MathPlugin plugin, which can be found in the core skills folder in the GitHub repository. It demonstrates how you can use the kernel_function and Annotated decorators to describe the function's input and output to planner.

@kernel_function(name="Add")
def add(
   self,
   number1: Annotated[float, "the first number to add"],
   number2: Annotated[float, "the second number to add"],
) -> Annotated[float, "the output is a float"]:
   return float(number1) + float(number2)

You can learn more about creating native functions in the Creating native functions section. In this article you'll learn the best practices for the following:

• How to create simple native functions with the SKFunction decorator
• Using multiple input parameters with native functions
• Calling nested functions from within native functions

Prompts

If plugins represent the "body" of your AI app, then prompts would represent the ears and mouth of your AI. They allow your AI app to listen to users asks and respond back with a natural language response.

To connect the ears and the mouth to the "brain," Semantic Kernel uses connectors. This allows you to easily swap out the AI services without rewriting code.

Below is an sample called Summarize that can be found in the samples folder in the GitHub repository.

[SUMMARIZATION RULES]
DONT WASTE WORDS
USE SHORT, CLEAR, COMPLETE SENTENCES.
DO NOT USE BULLET POINTS OR DASHES.
USE ACTIVE VOICE.
MAXIMIZE DETAIL, MEANING
FOCUS ON THE CONTENT

[BANNED PHRASES]
This article
This document
This page
This material
[END LIST]

Summarize:
Hello how are you?
+++++
Hello

Summarize this
{{$input}}
+++++

To semantically describe this function (as well as define the configuration for the AI service), you must also create a config.json file in the same folder as the prompt. This file describes the function's input parameters and description. Below is the config.json file for the Summarize function.

{
  "schema": 1,
  "description": "Summarize given text or any text document",
  "execution_settings": {
    "default": {
      "max_tokens": 512,
      "temperature": 0.0,
      "top_p": 0.0,
      "presence_penalty": 0.0,
      "frequency_penalty": 0.0
    }
  },
  "input_variables": [
    {
      "name": "input",
      "description": "Text to summarize",
      "default": "",
      "is_required": true
    }
  ]
}

Both description fields are used by planner, so it's important to provide a detailed, yet concise, description so the planner can make the best decision when orchestrating functions together. We recommend testing multiple descriptions to see which one works best for the widest range of scenarios.

You can learn more about creating prompts in the Creating prompts section. In this section you'll learn the best practices for the following:

• How to create prompts
• Adding input parameters with prompt templates
• Calling nested functions in prompts
• How to create files for your prompts

Take the next step

Now that you understand the basics of plugins, you can now go deeper into the details of creating semantic and native functions for your plugin.

Create a plugin with native functions