Overview of Plugins for Microsoft Copilot

Article
02/17/2024

Develop plugins that use the OpenAI schema to add custom functionality to Microsoft Copilot experiences, by connecting your own application data to Microsoft Copilot. Plugins enable a Copilot experience to interact with your own APIs, enhancing the experience to perform a wider range of actions. For more information on availability, see the Roadmap.

A plugin consists of an API, an API specification, and a plugin manifest.

Note

For Copilot for Microsoft 365, see Extend Microsoft Copilot for Microsoft 365.

Plugin capabilities

Plugins can retrieve information or perform tasks on behalf of a user. For example, a plugin could retrieve the latest news and weather information, or let a user add or remove items in a to-do list.

The large language model that powers Microsoft Copilot is trained on a wide range of data, which means that Microsoft Copilot can already understand and respond to a wide range of topics. However, Microsoft Copilot is unable to retrieve real-time information from third-party services. Plugins can extend the capabilities of Microsoft Copilot by retrieving real-time information, to allow users to interact with the information in a natural way.

Architecture of a plugin

A plugin is made up of three parts:

An API that runs on your own server, and which performs tasks.
An API specification, as a YAML or JSON file, which describes your API by using the OpenAPI schema. For example, openapi.yaml.
A plugin manifest, as a JSON file named ai-plugin.json, which is used by Microsoft Copilot to know when and how to use your plugin.

The API, API specification, and plugin manifest are all hosted on your own server:

Your web server, which hosts the API, the API spec, and the manifest

API

The part of your plugin that handles user actions is your API. Your API exposes the endpoints that enable users to achieve the tasks they need to achieve with your plugin - for example, a real estate plugin defining a /get-listings endpoint.

You can implement this API any way you want, using your preferred server-side programming language. Make sure to host your API on a server that's accessible to the internet via HTTPS.

API specification

You must document your API, by using a YAML or a JSON file that follows the OpenAPI schema - for example, openapi.yaml. Your API specification is required for Microsoft Copilot to know which endpoints your API exposes, how to send requests to them, and how to read responses.

Plugins don't directly display the result of their API calls to users. The Microsoft Copilot experience in which a plugin is installed reads the API responses by using the API specification, and then generates a user response based on the data that's returned by the API:

Flow diagram showing when a user asks a question, Microsoft Copilot calls the API, receives a response from it, and then presents the returned data to users

API specification fields

The following are the most important fields that your API specification must contain:

Field	Description	Type	Required	Guideline
`openapi`	The version of the OpenAPI schema that your specification is based on.	String	Yes	Use `3.0.1`
`info`	Metadata about your API.	Object	Yes	This object must contain the `title` and `description` strings. The `description` string must be less than 200 characters, and describes what the API does.
`servers`	The server where your API is available.	Array	Yes	This list must contain at least one object, with its `url` field set to the URL where your API is available. Initially, this is `http://localhost:8080`. Later in this tutorial, you'll change this URL when testing your plugin. You'll also need to change this URL once your plugin is available in production.
`paths`	The list of requests that your API can handle. In our example, there's just one request/path: `/get-listings`.	Object	Yes	Each request must have a `summary` field which describes what the request does. The `summary` string must be less than 200 characters. Each request must also have an `operationId` field.

Plugin manifest

The plugin manifest is a JSON file named ai-plugin.json, that contains metadata about your plugin. This manifest is used by Microsoft Copilot to know when and how to use your plugin.

Depending on the plugin description and depending on the question the user is asking, Microsoft Copilot can decide to invoke your plugin to respond to the user.

The manifest file doesn't contain code, but instead, contains a natural-language description of your plugin. The large language model that powers Microsoft Copilot reads your plugin description and can make decisions about whether your plugin can be used to assist a user with their question.

Here's an example manifest file for a real estate plugin that's available on contoso.com:

{
  "schema_version": "v1",
  "name_for_model": "contosorealestate",
  "description_for_model": "Plugin for finding properties for sale on the market in a particular city, and with a given number of bedrooms, bathrooms, and amenities",
  "name_for_human": "Contoso Real Estate",
  "description_for_human": "Find up to date and detailed real estate properties for sale on the market",
  "api": {
    "type": "openapi",
    "url": "https://contoso.com/openapi.yaml",
    "is_user_authenticated": false
  },
  "auth": {
    "type": "none"
  },
  "logo_url": "https://contoso.com/logo.png",
  "contact_email": "contact@contoso.com",
  "legal_info_url": "https://contoso.com/legal/",
  "privacy_policy_url": "https://contoso.com/privacy-policy"
}

Plugin manifest fields

The following fields can be used in a plugin manifest:

Field	Description	Type	Required?	Guideline
`schema_version`	Version of the schema this plugin manifest is written for.	String	Yes	Use `v1`.
`name_for_model`	Name of the plugin, for the model that powers Microsoft Copilot.	String	Yes	Must only contain letters; no spaces or special characters. Must be less than 50 characters.
`description_for_model`	Description of the plugin, for the model that powers Microsoft Copilot.	String	Yes	Limited to 8000 characters. This is a factual description of your plugin. Describe what your plugin does for users. Don't use this description to tell the model to use your plugin or to answer in a particular way.
`name_for_human`	Name of the plugin, displayed to users in the Microsoft Copilot experience where your plugin is installed.	String	Yes	Limited to 20 characters.
`description_for_human`	Description of the plugin, displayed to users in the Microsoft Copilot experience where your plugin is installed.	String	Yes	Limited to 100 characters.
`api`	Information about where to find your API documentation.	Object	Yes	A JSON object that has `type`, `url`, and `is_user_authenticated` properties. The `type` property must be set to `openapi`. The `url` property must point to the location of your API's OpenAPI documentation YAML file (`openapi.yaml`, in this example).
`auth`	Information about whether or not your API requires authentication from Microsoft Copilot.	Object	Yes	To learn more, see Restrict access to your plugin API.
`logo_url`	The URL to your plugin's logo.	String	Yes	The URL must point to an image resource. The image must be 512x512 pixels. The image must not be a GIF.
`contact_email`	Email address for support requests about your plugin.	String	Yes	This address is publicly visible.
`legal_info_url`	URL of the terms of use, terms of service, or end user license for your plugin. The terms are directly between you and the end user.	String	Yes	This URL is publicly visible.
`privacy_policy_url`	A link to your privacy policy, which must describe your usage of any data that's collected by or processed through the plugin.	String	Yes	This URL is publicly visible.

Plugin process flow

Microsoft Copilot uses the following process flow when the user asks a question and Microsoft Copilot answers the question by searching for and using a plugin:

Diagram showing the process flow for the user asking a question and Copilot answering the question by searching for and using a plugin

The process flow has the following steps. The Examples column below refers to an example plugin that can find real estate listings based on search criteria such as the city. To learn to build this real estate plugin example yourself, see Get started building a plugin for Microsoft Copilot.

#	Steps	Examples
1	The user asks a question in the chat UI of the Microsoft Copilot experience.	"Find 4 bedroom properties for sale in New York"
2	Microsoft Copilot evaluates the question and decides whether to use a plugin to help answer the question, based on the plugin description and API specification.	The description of the plugin is "Plugin for finding properties for sale on the market in a particular city, and with a given number of bedrooms, bathrooms, and amenities". The plugin's API exposes a single endpoint called `get-listings`, which is described as "Get a list of properties matching the specified criteria".
3	Microsoft Copilot calls the plugin API, by using the plugin's OpenAPI specification to generate the right API request.	Microsoft Copilot sends an API request to `https://contoso.com/get-listings?city=New+York&bedrooms=4`.
4	The plugin API receives the request and returns a response.	The API returns data such as `[{"title": "Luminous apartment in the heart of the city", "city": "New York", "bedrooms": 4, "description": "..."}]`.
5	Microsoft Copilot extracts the information from the response, by using the plugin's OpenAPI specification.	n/a
6	Microsoft Copilot answers the user's question, by using its training data, its natural language generation abilities, and the information that's in the API response.	"Here are a few apartments for sale in New York that have 4 bedrooms: ..."