聊天套件

agent-framework-chatkit 將 OpenAI ChatKit 執行緒項目轉換為代理框架訊息,並將串流代理更新轉回 ChatKit 事件。 當你想要 ChatKit 前端搭配 Agent Framework Python 後端時,可以用它。

整合提供:

  • ThreadItemConverter 用於轉換 ChatKit 的執行緒項目與附件。
  • stream_agent_response() 用於將串流客服更新轉換為 ChatKit 事件。
  • simple_to_agent_input() 用於預設訊息轉換路徑。

Prerequisites

  • Python 3.11 或更新版本。
  • 後端網頁框架如 FastAPI。
  • 用於 ChatKit 前端的 Node.js。
  • 已部署模型的 Microsoft Foundry 專案
  • Azure CLI 驗證(az login)。
  • 一個用於生產前端網域的 ChatKit 網域金鑰。

安裝套件

pip install agent-framework-chatkit agent-framework-foundry --pre

配置 Foundry 模型

此範例使用 FoundryChatClient 和 AzureCliCredential 建立代理程式。 在開始後端之前,先設定專案端點和模型部署:

export FOUNDRY_PROJECT_ENDPOINT="https://<account>.services.ai.azure.com/api/projects/<project>"
export FOUNDRY_MODEL="<model-deployment>"

建立 ChatKit 伺服器

子類 ChatKitServer,建立 Agent Framework 代理,並設定執行緒項目與附件的轉換器。


This server integrates Agent Framework agents with ChatKit's server protocol,
providing weather information with interactive widgets and time queries through Azure OpenAI.
"""

def __init__(self, data_store: SQLiteStore, attachment_store: FileBasedAttachmentStore):
    super().__init__(data_store, attachment_store)

    logger.info("Initializing WeatherChatKitServer")

    # Create Agent Framework agent with Azure OpenAI
    # For authentication, run `az login` command in terminal
    try:
        self.weather_agent = Agent(
            client=FoundryChatClient(credential=AzureCliCredential()),
            instructions=(
                "You are a helpful weather assistant with image analysis capabilities. "
                "You can provide weather information for any location, tell the current time, "
                "and analyze images that users upload. Be friendly and informative in your responses.\n\n"
                "If a user asks to see a list of cities or wants to choose from available cities, "
                "use the show_city_selector tool to display an interactive city selector.\n\n"
                "When users upload images, you will automatically receive them and can analyze their content. "
                "Describe what you see in detail and be helpful in answering questions about the images."
            ),
            tools=[get_weather, get_time, show_city_selector],
        )
        logger.info("Weather agent initialized successfully with Azure OpenAI")
    except Exception as e:
        logger.error(f"Failed to initialize weather agent: {e}")
        raise

    # Create ThreadItemConverter with attachment data fetcher
    self.converter = ThreadItemConverter(
        attachment_data_fetcher=self._fetch_attachment_data,
    )

    logger.info("WeatherChatKitServer initialized")

轉換與串流回應

載入執行緒歷史,轉換成 Agent Framework 訊息,將代理程式切換成串流模式,並產生 ChatKit 事件。

    thread: ThreadMetadata,
    input_user_message: UserMessageItem | None,
    context: dict[str, Any],
) -> AsyncIterator[ThreadStreamEvent]:
    """Handle incoming user messages and generate responses.

    This method converts ChatKit messages to Agent Framework format using ThreadItemConverter,
    runs the agent, converts the response back to ChatKit events using stream_agent_response,
    and creates interactive weather widgets when weather data is queried.
    """
    if input_user_message is None:
        logger.debug("Received None user message, skipping")
        return

    logger.info(f"Processing message for thread: {thread.id}")

    try:
        # Collect weather data from get_weather and track the city selector flag for this request
        weather_results: list[WeatherData] = []
        show_city_selector = False

        # Load full thread history from the store
        thread_items_page = await self.store.load_thread_items(
            thread_id=thread.id,
            after=None,
            limit=1000,
            order="asc",
            context=context,
        )
        thread_items = thread_items_page.data

        # Convert ALL thread items to Agent Framework ChatMessages using ThreadItemConverter
        # This ensures the agent has the full conversation context
        agent_messages = await self.converter.to_agent_input(thread_items)

        if not agent_messages:
            logger.warning("No messages after conversion")
            return

        logger.info(f"Running agent with {len(agent_messages)} message(s)")

        # Run the Agent Framework agent with streaming. get_weather appends to weather_results.
        agent_stream = self.weather_agent.run(
            agent_messages,
            stream=True,
            function_invocation_kwargs={"weather_results": weather_results},
        )

        # Create an intercepting stream that detects the city selector marker while passing through updates
        async def intercept_stream() -> AsyncIterator[AgentResponseUpdate]:
            nonlocal show_city_selector
            async for update in agent_stream:
                # Check for the city selector marker in function results
                for content in update.contents:
                    if content.type == "function_result" and content.result == "__SHOW_CITY_SELECTOR__":
                        show_city_selector = True
                        logger.info("City selector flag detected")
                yield update

        # Stream updates as ChatKit events with interception
        async for event in stream_agent_response(
            intercept_stream(),
            thread_id=thread.id,
        ):
            yield event

        # If weather data was collected during the tool call, create a widget
        if weather_results:
            weather_data = weather_results[-1]
            logger.info(f"Creating weather widget for location: {weather_data.location}")
            # Create weather widget
            widget = render_weather_widget(weather_data)
            copy_text = weather_widget_copy_text(weather_data)

            # Stream the widget
            async for widget_event in stream_widget(thread_id=thread.id, widget=widget, copy_text=copy_text):

先呼叫 agent.run(..., stream=True) ,然後把回應串流和 ChatKit 執行緒 ID 傳給 stream_agent_response(...)。 不要把代理本身直接交給轉換器。

ThreadItemConverter 保留 StructuredInputItem 狀態,並將每個問題在使用者訊息中呈現為已回答、跳過或未回答。 覆蓋非同步 structured_input_to_input() 方法以自訂格式或遮蔽答案。 回傳一個 Message訊息清單,或 None 省略該項目。

轉換器也會在後續回合保留已完成 GeneratedImageItem 的數值作為模型上下文。 預設轉換會建立一個帶有簡短文字前言及生成圖片 URI 的使用者 Message 。 資料 URI 保留其嵌入的媒體類型,外部 URL 使用 image/*,且未完成的項目若無影像會被跳過。

覆寫非同步generated_image_to_input()方法,以自訂產生的影像內容,例如解析私人影像 URL。 回傳一個 Message訊息清單,或 None 省略該項目。

完整範例還展示了 SQLite 支援的執行緒、檔案上傳、附件儲存、動作以及互動式小工具。

Warning

ChatKit 前端是從 OpenAI 的 CDN 載入,並向 OpenAI 網域發出外撥請求。 目前無法自行託管,也不適用於實體隔離斷網的環境。

下一步