agent-framework-chatkit 将 OpenAI ChatKit 线程条目转换为 Agent Framework 消息,并将流式传输的代理更新转换回 ChatKit 事件。 当想要将 ChatKit 前端与 Agent Framework Python后端配合使用时使用。
集成提供:
-
ThreadItemConverter用于转换 ChatKit 线程项和附件。 -
stream_agent_response()用于将流式代理更新转换为 ChatKit 事件。 -
simple_to_agent_input()用于默认消息转换路径。
Prerequisites
- Python 3.11 或更高版本。
- 后端 Web 框架,如 FastAPI。
- 用于 ChatKit 前端的 Node.js。
- 具有已部署模型的 Microsoft Foundry 项目。
- Azure CLI身份验证(
az login)。 - 生产前端域的 ChatKit 域密钥。
安装软件包
pip install agent-framework-chatkit agent-framework-foundry --pre
配置 Foundry 模型
该示例使用 AzureCliCredential 和 FoundryChatClient 创建代理。
在启动后端之前设置项目终结点和模型部署:
export FOUNDRY_PROJECT_ENDPOINT="https://<account>.services.ai.azure.com/api/projects/<project>"
export FOUNDRY_MODEL="<model-deployment>"
创建 ChatKit 服务器
子类 ChatKitServer,创建 Agent Framework 代理,并为线程项和附件配置转换器。
This server integrates Agent Framework agents with ChatKit's server protocol,
providing weather information with interactive widgets and time queries through Azure OpenAI.
"""
def __init__(self, data_store: SQLiteStore, attachment_store: FileBasedAttachmentStore):
super().__init__(data_store, attachment_store)
logger.info("Initializing WeatherChatKitServer")
# Create Agent Framework agent with Azure OpenAI
# For authentication, run `az login` command in terminal
try:
self.weather_agent = Agent(
client=FoundryChatClient(credential=AzureCliCredential()),
instructions=(
"You are a helpful weather assistant with image analysis capabilities. "
"You can provide weather information for any location, tell the current time, "
"and analyze images that users upload. Be friendly and informative in your responses.\n\n"
"If a user asks to see a list of cities or wants to choose from available cities, "
"use the show_city_selector tool to display an interactive city selector.\n\n"
"When users upload images, you will automatically receive them and can analyze their content. "
"Describe what you see in detail and be helpful in answering questions about the images."
),
tools=[get_weather, get_time, show_city_selector],
)
logger.info("Weather agent initialized successfully with Azure OpenAI")
except Exception as e:
logger.error(f"Failed to initialize weather agent: {e}")
raise
# Create ThreadItemConverter with attachment data fetcher
self.converter = ThreadItemConverter(
attachment_data_fetcher=self._fetch_attachment_data,
)
logger.info("WeatherChatKitServer initialized")
转换并流式传输响应
加载线程历史记录,将其转换为代理框架消息,在流式处理模式下运行代理,并生成 ChatKit 事件。
thread: ThreadMetadata,
input_user_message: UserMessageItem | None,
context: dict[str, Any],
) -> AsyncIterator[ThreadStreamEvent]:
"""Handle incoming user messages and generate responses.
This method converts ChatKit messages to Agent Framework format using ThreadItemConverter,
runs the agent, converts the response back to ChatKit events using stream_agent_response,
and creates interactive weather widgets when weather data is queried.
"""
if input_user_message is None:
logger.debug("Received None user message, skipping")
return
logger.info(f"Processing message for thread: {thread.id}")
try:
# Collect weather data from get_weather and track the city selector flag for this request
weather_results: list[WeatherData] = []
show_city_selector = False
# Load full thread history from the store
thread_items_page = await self.store.load_thread_items(
thread_id=thread.id,
after=None,
limit=1000,
order="asc",
context=context,
)
thread_items = thread_items_page.data
# Convert ALL thread items to Agent Framework ChatMessages using ThreadItemConverter
# This ensures the agent has the full conversation context
agent_messages = await self.converter.to_agent_input(thread_items)
if not agent_messages:
logger.warning("No messages after conversion")
return
logger.info(f"Running agent with {len(agent_messages)} message(s)")
# Run the Agent Framework agent with streaming. get_weather appends to weather_results.
agent_stream = self.weather_agent.run(
agent_messages,
stream=True,
function_invocation_kwargs={"weather_results": weather_results},
)
# Create an intercepting stream that detects the city selector marker while passing through updates
async def intercept_stream() -> AsyncIterator[AgentResponseUpdate]:
nonlocal show_city_selector
async for update in agent_stream:
# Check for the city selector marker in function results
for content in update.contents:
if content.type == "function_result" and content.result == "__SHOW_CITY_SELECTOR__":
show_city_selector = True
logger.info("City selector flag detected")
yield update
# Stream updates as ChatKit events with interception
async for event in stream_agent_response(
intercept_stream(),
thread_id=thread.id,
):
yield event
# If weather data was collected during the tool call, create a widget
if weather_results:
weather_data = weather_results[-1]
logger.info(f"Creating weather widget for location: {weather_data.location}")
# Create weather widget
widget = render_weather_widget(weather_data)
copy_text = weather_widget_copy_text(weather_data)
# Stream the widget
async for widget_event in stream_widget(thread_id=thread.id, widget=widget, copy_text=copy_text):
首先调用 agent.run(..., stream=True) ,然后将该响应流和 ChatKit 线程 ID 传递给 stream_agent_response(...)。 不要将代理本身传递给转换器。
ThreadItemConverter保留StructuredInputItem的状态,并在用户消息中将每个问题表示为已回答、已跳过或未回答。 替代异步 structured_input_to_input() 方法,以自定义格式或对答案进行编辑。 返回一个 Message消息列表,或 None 省略该项。
转换器还会将已完成的 GeneratedImageItem 值保留为后续轮次中的模型上下文。 默认转换创建一个带有短文本前言和生成的图像 URI 的用户 Message 。 数据 URI 会保留其嵌入的媒体类型,外部 URL 使用 image/*,而没有图像的未完成项目会被跳过。
重写异步 generated_image_to_input() 方法,以自定义生成图像时的上下文,例如解析私有图像 URL。 返回一个 Message消息列表,或 None 省略该项。
完整示例还演示了 SQLite 支持的线程、文件上传、附件存储、操作和交互式小组件。
Warning
ChatKit 前端从 OpenAI 的 CDN 加载,并向 OpenAI 域名发起对外请求。 它目前无法自托管,也不适用于与外部网络隔离的环境。