Share via

An Overview of the Agent Architecture

  • Article

Important

AgentChat patterns are in the experimental stage. These patterns are under active development and may change significantly before advancing to the preview or release candidate stage.

This article covers key concepts in the architecture of the Agent Framework, including foundational principles, design objectives, and strategic goals.

Goals

The Agent Framework was developed with the following key priorities in mind:

  • The Semantic Kernel agent framework serves as the core foundation for implementing agent functionalities.
  • Multiple agents can collaborate within a single conversation, while integrating human input.
  • An agent can engage in and manage multiple concurrent conversations simultaneously.
  • Different types of agents can participate in the same conversation, each contributing their unique capabilities.

Agent

The abstract Agent class serves as the core abstraction for all types of agents, providing a foundational structure that can be extended to create more specialized agents. One key subclass is Kernel Agent, which establishes a direct association with a Kernel object. This relationship forms the basis for more specific agent implementations, such as the ChatCompletionAgent, OpenAIAssistantAgent, AzureAIAgent, or OpenAIResponsesAgent, all of which leverage the Kernel's capabilities to execute their respective functions.

The underlying Semantic Kernel Agent abstraction can be found here:

Agents are currently unavailable in Java.

Agents can either be invoked directly to perform tasks or orchestrated within an AgentChat, where multiple agents may collaborate or interact dynamically with user inputs. This flexible structure allows agents to adapt to various conversational or task-driven scenarios, providing developers with robust tools for building intelligent, multi-agent systems.

Deep Dive:

Agent Thread

The abstract AgentThread class serves as the core abstraction for threads or conversation state. It abstracts away the different ways in which conversation state may be managed for different agents.

Stateful agent services often store conversation state in the service, and you can interact with it via an id. Other agents may require the entire chat history to be passed to the agent on each invocation, in which case the conversation state is managed locally in the application.

Stateful agents typically only work with a matching AgentThread implementation, while other types of agents could work with more than one AgentThread type. For example, AzureAIAgent requires a matching AzureAIAgentThread. This is because the Azure AI Agent service stores conversations in the service, and requires specific service calls to create a thread and update it. If a different agent thread type is used with the AzureAIAgent, we fail fast due to an unexpected thread type and raise an exception to alert the caller.

Agent Chat

The AgentChat class serves as the foundational component that enables agents of any type to engage in a specific conversation. This class provides the essential capabilities for managing agent interactions within a chat environment. Building on this, the AgentGroupChat class extends these capabilities by offering a strategy-based container, which allows multiple agents to collaborate across numerous interactions within the same conversation.

Important

The current OpenAIResponsesAgent is not supported as part of Semantic Kernel's AgentGroupChat patterns. Stayed tuned for updates.

This structure facilitates more complex, multi-agent scenarios where different agents can work together, share information, and dynamically respond to evolving conversations, making it an ideal solution for advanced use cases such as customer support, multi-faceted task management, or collaborative problem-solving environments.

Deep Dive:

Agent Channel

The Agent Channel class enables agents of various types to participate in an AgentChat. This functionality is completely hidden from users of the Agent Framework and only needs to be considered by developers creating a custom Agent.

Agents are currently unavailable in Java.

Agent Alignment with Semantic Kernel Features

The Agent Framework is built on the foundational concepts and features that many developers have come to know within the Semantic Kernel ecosystem. These core principles serve as the building blocks for the Agent Framework’s design. By leveraging the familiar structure and capabilities of the Semantic Kernel, the Agent Framework extends its functionality to enable more advanced, autonomous agent behaviors, while maintaining consistency with the broader Semantic Kernel architecture. This ensures a smooth transition for developers, allowing them to apply their existing knowledge to create intelligent, adaptable agents within the framework.

The Kernel

At the heart of the Semantic Kernel ecosystem is the Kernel, which serves as the core object that drives AI operations and interactions. To create any agent within this framework, a Kernel instance is required as it provides the foundational context and capabilities for the agent’s functionality. The Kernel acts as the engine for processing instructions, managing state, and invoking the necessary AI services that power the agent's behavior.

At the core of the Semantic Kernel ecosystem is the Kernel, the primary object responsible for managing AI operations and interactions. To simplify onboarding, the Kernel is optional—if none is supplied when constructing an agent, a new Kernel instance is automatically created for the caller. For more advanced scenarios, such as applying filters, the caller must configure the desired filters on a Kernel instance and explicitly pass it to the agent.

Agents are currently unavailable in Java.

The AzureAIAgent, ChatCompletionAgent, OpenAIAssistantAgent, and OpenAIResponsesAgent articles provide specific details on how to create each type of agent.

These resources offer step-by-step instructions and highlight the key configurations needed to tailor the agents to different conversational or task-based applications, demonstrating how the Kernel enables dynamic and intelligent agent behaviors across diverse use cases.

Agents are currently unavailable in Java.

Plugins and Function Calling

Plugins are a fundamental aspect of the Semantic Kernel, enabling developers to integrate custom functionalities and extend the capabilities of an AI application. These plugins offer a flexible way to incorporate specialized features or business-specific logic into the core AI workflows. Additionally, agent capabilities within the framework can be significantly enhanced by utilizing Plugins and leveraging Function Calling. This allows agents to dynamically interact with external services or execute complex tasks, further expanding the scope and versatility of the AI system within diverse applications.

Example:

Agents are currently unavailable in Java.

Agent Messages

Agent messaging, including both input and response, is built upon the core content types of the Semantic Kernel, providing a unified structure for communication. This design choice simplifies the process of transitioning from traditional chat-completion patterns to more advanced agent-driven patterns in your application development. By leveraging familiar Semantic Kernel content types, developers can seamlessly integrate agent capabilities into their applications without needing to overhaul existing systems. This streamlining ensures that as you evolve from basic conversational AI to more autonomous, task-oriented agents, the underlying framework remains consistent, making development faster and more efficient.

Note: The OpenAIAssistantAgent introduced content types specific to its usage for File References and Content Annotation:

Agents are currently unavailable in Java.

Templating

An agent's role is primarily shaped by the instructions it receives, which dictate its behavior and actions. Similar to invoking a Kernel prompt, an agent's instructions can include templated parameters—both values and functions—that are dynamically substituted during execution. This enables flexible, context-aware responses, allowing the agent to adjust its output based on real-time input.

Additionally, an agent can be configured directly using a Prompt Template Configuration, providing developers with a structured and reusable way to define its behavior. This approach offers a powerful tool for standardizing and customizing agent instructions, ensuring consistency across various use cases while still maintaining dynamic adaptability.

Example:

Agents are currently unavailable in Java.

Chat Completion

The ChatCompletionAgent is designed around any Semantic Kernel AI service, offering a flexible and convenient persona encapsulation that can be seamlessly integrated into a wide range of applications. This agent allows developers to easily bring conversational AI capabilities into their systems without having to deal with complex implementation details. It mirrors the features and patterns found in the underlying AI service, ensuring that all functionalities—such as natural language processing, dialogue management, and contextual understanding—are fully supported within the ChatCompletionAgent, making it a powerful tool for building conversational interfaces.

Agents are currently unavailable in Java.

Explore the Common Agent Invocation API