Share via

Configuring Agents with Semantic Kernel Plugins

Important

This feature is in the release candidate stage. Features at this stage are nearly complete and generally stable, though they may undergo minor refinements or optimizations before reaching full general availability.

Functions and Plugins in Semantic Kernel

Function calling is a powerful tool that allows developers to add custom functionalities and expand the capabilities of AI applications. The Semantic Kernel Plugin architecture offers a flexible framework to support Function Calling. For an Agent, integrating Plugins and Function Calling is built on this foundational Semantic Kernel feature.

Once configured, an agent will choose when and how to call an available function, as it would in any usage outside of the Agent Framework.

Tip

API reference:

Tip

API reference:

Tip

API reference:

Adding Plugins to an Agent

Any Plugin available to an Agent is managed within its respective Kernel instance. This setup enables each Agent to access distinct functionalities based on its specific role.

Plugins can be added to the Kernel either before or after the Agent is created. The process of initializing Plugins follows the same patterns used for any Semantic Kernel implementation, allowing for consistency and ease of use in managing AI capabilities.

Note

For a ChatCompletionAgent, the function calling mode must be explicitly enabled. OpenAIAssistant agent is always based on automatic function calling.

// Factory method to product an agent with a specific role.
// Could be incorporated into DI initialization.
ChatCompletionAgent CreateSpecificAgent(Kernel kernel, string credentials)
{
    // Clone kernel instance to allow for agent specific plug-in definition
    Kernel agentKernel = kernel.Clone();

    // Import plug-in from type
    agentKernel.ImportPluginFromType<StatelessPlugin>();

    // Import plug-in from object
    agentKernel.ImportPluginFromObject(new StatefulPlugin(credentials));

    // Create the agent
    return
        new ChatCompletionAgent()
        {
            Name = "<agent name>",
            Instructions = "<agent instructions>",
            Kernel = agentKernel,
            Arguments = new KernelArguments(
                new OpenAIPromptExecutionSettings()
                {
                    FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
                })
        };
}

There are two ways to create a ChatCompletionAgent with plugins.

Method 1: Specify Plugins via the Constructor

You can directly pass a list of plugins to the constructor:

from semantic_kernel.agents import ChatCompletionAgent

# Create the Chat Completion Agent instance by specifying a list of plugins
agent = ChatCompletionAgent(
    service=AzureChatCompletion(),
    instructions="<instructions>",
    plugins=[SamplePlugin()]
)

Tip

By default, auto-function calling is enabled. To disable it, set the function_choice_behavior argument to function_choice_behavior=FunctionChoiceBehavior.Auto(auto_invoke=False) in the constructor. With this setting, plugins are broadcast to the model, but they are not automatically invoked. If execution settings specify the same service_id or ai_model_id as the AI service configuration, the function calling behavior defined in the execution settings (via KernelArguments) will take precedence over the function choice behavior set in the constructor.

Method 2: Configure the Kernel Manually

If no kernel is provided via the constructor, one is automatically created during model validation. Any plugins passed in take precedence and are added to the kernel. For more fine-grained control over the kernel's state, follow these steps:

from semantic_kernel.agents import ChatCompletionAgent
from semantic_kernel.connectors.ai import FunctionChoiceBehavior
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureChatPromptExecutionSettings
from semantic_kernel.functions import KernelFunctionFromPrompt
from semantic_kernel.kernel import Kernel

# Create the instance of the Kernel
kernel = Kernel()

# Add the chat completion service to the Kernel
kernel.add_service(AzureChatCompletion())

# Get the AI Service settings
settings = kernel.get_prompt_execution_settings_from_service_id()

# Configure the function choice behavior to auto invoke kernel functions
settings.function_choice_behavior = FunctionChoiceBehavior.Auto()

# Add the Plugin to the Kernel
kernel.add_plugin(SamplePlugin(), plugin_name="<plugin name>")

# Create the agent
agent = ChatCompletionAgent(
    kernel=kernel,
    name=<agent name>,
    instructions=<agent instructions>,
    arguments=KernelArguments(settings=settings),
)

Tip

If a service_id is not specified when adding a service to the kernel, it defaults to default. When configuring multiple AI services on the kernel, it’s recommended to differentiate them using the service_id argument. This allows you to retrieve execution settings for a specific service_id and tie those settings to the desired service.

var chatCompletion = OpenAIChatCompletion.builder()
    .withModelId("<model-id>")
    .withOpenAIAsyncClient(new OpenAIClientBuilder()
            .credential(new AzureKeyCredential("<api-key>"))
            .endpoint("<endpoint>")
            .buildAsyncClient())
    .build();

Kernel kernel = Kernel.builder()
    .withAIService(ChatCompletionService.class, chatCompletion)
    .withPlugin(KernelPluginFactory.createFromObject(new SamplePlugin(), "<plugin name>"))
    .build();

var agent = ChatCompletionAgent.builder()
    .withKernel(kernel)
    .withName("<agent name>")
    .withInstructions("<agent instructions>")
    .build();

Adding Functions to an Agent

A Plugin is the most common approach for configuring Function Calling. However, individual functions can also be supplied independently including prompt functions.

// Factory method to product an agent with a specific role.
// Could be incorporated into DI initialization.
ChatCompletionAgent CreateSpecificAgent(Kernel kernel)
{
    // Clone kernel instance to allow for agent specific plug-in definition
    Kernel agentKernel = kernel.Clone();

    // Create plug-in from a static function
    var functionFromMethod = agentKernel.CreateFunctionFromMethod(StatelessPlugin.AStaticMethod);

    // Create plug-in from a prompt
    var functionFromPrompt = agentKernel.CreateFunctionFromPrompt("<your prompt instructions>");

    // Add to the kernel
    agentKernel.ImportPluginFromFunctions("my_plugin", [functionFromMethod, functionFromPrompt]);

    // Create the agent
    return
        new ChatCompletionAgent()
        {
            Name = "<agent name>",
            Instructions = "<agent instructions>",
            Kernel = agentKernel,
            Arguments = new KernelArguments(
                new OpenAIPromptExecutionSettings()
                {
                    FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
                })
        };
}

from semantic_kernel.agents import ChatCompletionAgent
from semantic_kernel.connectors.ai import FunctionChoiceBehavior
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureChatPromptExecutionSettings
from semantic_kernel.functions import KernelFunctionFromPrompt
from semantic_kernel.kernel import Kernel

# Create the instance of the Kernel
kernel = Kernel()

# Add the chat completion service to the Kernel
kernel.add_service(AzureChatCompletion())

# Create the AI Service settings
settings = AzureChatPromptExecutionSettings()

# Configure the function choice behavior to auto invoke kernel functions
settings.function_choice_behavior = FunctionChoiceBehavior.Auto()

# Add the Plugin to the Kernel
kernel.add_function(
    plugin_name="<plugin_name>",
    function=KernelFunctionFromPrompt(
        function_name="<function_name>",
        prompt="<your prompt instructions>",
    )
)

# Create the agent
agent = ChatCompletionAgent(
    kernel=kernel,
    name=<agent name>,
    instructions=<agent instructions>,
    arguments=KernelArguments(settings=settings),
)

var chatCompletion = OpenAIChatCompletion.builder()
    .withModelId("<model-id>")
    .withOpenAIAsyncClient(new OpenAIClientBuilder()
            .credential(new AzureKeyCredential("<api-key>"))
            .endpoint("<endpoint>")
            .buildAsyncClient())
    .build();

// Create function from method
var functionFromMethod = KernelFunction.createFromMethod(SamplePlugin.class.getMethod("method"), new SamplePlugin());

// Create function from prompt
var functionFromPrompt = KernelFunction.createFromPrompt("<your prompt instructions>");

// Create the kernel with a plugin from the two functions
Kernel kernel = Kernel.builder()
    .withAIService(ChatCompletionService.class, chatCompletion)
    .withPlugin(KernelPluginFactory.createFromFunctions("SamplePlugin", List.of(functionFromMethod, functionFromPrompt)))
    .build();

InvocationContext invocationContext = InvocationContext.builder()
    .withFunctionChoiceBehavior(FunctionChoiceBehavior.auto(true))
    .build();

// Create the agent
var agent = ChatCompletionAgent.builder()
    .withKernel(kernel)
    .withName("<agent name>")
    .withInstructions("<agent instructions>")
    .withInvocationContext(invocationContext)
    .build();

Limitations for Agent Function Calling

When directly invoking aChatCompletionAgent, all Function Choice Behaviors are supported. However, when using an OpenAIAssistant, only Automatic Function Calling is currently available.

How-To

For an end-to-end example for using function calling, see:

Next steps

How to Stream Agent Responses