Deploy and operate your migrated Agent Framework workflow

Warning

Prompt Flow feature development ended on April 20, 2026. The feature will be fully retired on April 20, 2027. On the retirement date, Prompt Flow enters read-only mode. Your existing flows will continue to operate until that date.

Recommended action: Migrate your Prompt Flow workloads to Microsoft Agent Framework before April 20, 2027.

This article covers the operations and cutover steps of the Prompt Flow to Microsoft Agent Framework migration: setting up tracing, deploying to Azure Container Apps, adding a CI/CD quality gate, and cutting over production traffic. For the audit, rebuild, and validation steps, see Rebuild and validate your Prompt Flow workflow in Microsoft Agent Framework.

Prerequisites

Completed the audit, rebuild, and validation steps with a mean parity score ≥ 3.5 in parity_results.csv.
An Application Insights instance.
An Azure Container Registry and Container Apps environment.
A GitHub repository with secrets configured for CI/CD.

Install additional packages:

pip install azure-monitor-opentelemetry fastapi uvicorn

Migrate operations

This step replaces the operational infrastructure that Prompt Flow managed automatically.

Sub-phase	Replaces	Details
4a — Tracing	Prompt Flow built-in run viewer	OpenTelemetry + Application Insights
4b — Deployment	Prompt Flow Managed Online Endpoint	FastAPI + Azure Container Apps
4c — CI/CD	Manual evaluation runs in the Prompt Flow UI	GitHub Actions quality gate

Set up tracing with OpenTelemetry

Agent Framework automatically emits OpenTelemetry spans for every executor invocation, agent call, and LLM request. Connect these to Application Insights by calling configure_azure_monitor() once at application startup.

Add the Application Insights connection string to your .env file:

APPLICATIONINSIGHTS_CONNECTION_STRING=<your-connection-string>

Call configure_azure_monitor() before any workflow.run() call:

import os

from dotenv import load_dotenv
from azure.monitor.opentelemetry import configure_azure_monitor

load_dotenv()

configure_azure_monitor(
    connection_string=os.environ[
        "APPLICATIONINSIGHTS_CONNECTION_STRING"
    ]
)

# All workflow.run() calls after this point emit traces.

View traces in the Azure portal: go to your Application Insights resource and select Transaction Search.

Important

Call configure_azure_monitor() at application startup, not inside a handler or after workflow.run(). Traces emitted before the call are lost.

Deploy to Azure Container Apps

Wrap your Agent Framework workflow in a FastAPI service and deploy it as a container.

Create the FastAPI wrapper

Create an app.py file that exposes a /ask endpoint:

"""Wraps an Agent Framework workflow in a FastAPI service."""

import os
from contextlib import asynccontextmanager

from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from azure.monitor.opentelemetry import configure_azure_monitor

load_dotenv()

# Import your rebuilt workflow
from your_workflow_module import workflow

# Configure tracing if connection string is present.
appinsights_conn = os.getenv("APPLICATIONINSIGHTS_CONNECTION_STRING")
if appinsights_conn:
    configure_azure_monitor(connection_string=appinsights_conn)


class QuestionRequest(BaseModel):
    question: str


class AnswerResponse(BaseModel):
    answer: str


@asynccontextmanager
async def lifespan(app: FastAPI):
    yield


app = FastAPI(title="Agent Framework Workflow Service", lifespan=lifespan)


@app.get("/health")
async def health():
    return {"status": "ok"}


@app.post("/ask", response_model=AnswerResponse)
async def ask(payload: QuestionRequest):
    if not payload.question.strip():
        raise HTTPException(
            status_code=400, detail="Question must not be empty."
        )

    result = await workflow.run(payload.question.strip())
    outputs = result.get_outputs()

    if not outputs:
        raise HTTPException(
            status_code=500,
            detail="Workflow produced no output.",
        )

    return AnswerResponse(answer=outputs[0])

Test locally with:

uvicorn app:app --reload

Create the Dockerfile

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

Deploy with Azure CLI

Build and push the container image:

az acr build \
  --registry <your-acr> \
  --image maf-app:latest \
  .

Create the Container App:

az containerapp create \
  --name maf-app \
  --resource-group <your-rg> \
  --environment <your-env> \
  --image <your-acr>.azurecr.io/maf-app:latest \
  --target-port 8000 \
  --ingress external \
  --registry-server <your-acr>.azurecr.io \
  --secrets openai-key="$AZURE_OPENAI_API_KEY" \
  --env-vars \
    AZURE_OPENAI_API_KEY=secretref:openai-key \
    AZURE_OPENAI_ENDPOINT="https://<resource>.openai.azure.com/" \
    AZURE_OPENAI_CHAT_DEPLOYMENT_NAME="<deployment>"

Verify the deployment:

APP_URL=$(az containerapp show \
  --name maf-app \
  --resource-group <your-rg> \
  --query "properties.configuration.ingress.fqdn" -o tsv)

curl "https://${APP_URL}/health"

Tip

For production deployments, use managed identity instead of API keys. Pass credential=ManagedIdentityCredential() to AzureOpenAIChatClient() and remove AZURE_OPENAI_API_KEY from your environment variables. Use Key Vault secret references (secretref:kv-*) for any remaining secrets.

Add a CI/CD quality gate

Create a GitHub Actions workflow that runs the parity check on every push and fails the pipeline if the mean similarity drops below the threshold.

name: Evaluate on Deploy
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  evaluate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
          cache: pip

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Run parity evaluation
        run: python parity_check.py
        env:
          AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
          AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
          AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_CHAT_DEPLOYMENT_NAME }}

      - name: Enforce quality gate
        run: |
          python -c "
          import pandas as pd, sys
          df = pd.read_csv('parity_results.csv')
          score = df['similarity'].mean()
          print(f'Mean similarity: {score:.2f} / 5.0')
          sys.exit(0 if score >= 3.5 else 1)
          "

Required repository secrets:

AZURE_OPENAI_API_KEY
AZURE_OPENAI_ENDPOINT
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME

Cut over

Switch production traffic to Agent Framework and decommission Prompt Flow resources.

Pre-cutover checklist

Verify all of the following before proceeding:

Mean parity score ≥ 3.5 across the full test suite.
Agent Framework Container App is healthy (az containerapp show).
Tracing is confirmed in Application Insights.
CI/CD quality gate is passing on the main branch.
API gateway or client configuration is updated to point at the Agent Framework endpoint.

Run the cutover

Archive your existing Prompt Flow YAML:

cp -r <your-flow-directory> ./archived-flow/

Delete the Prompt Flow managed online endpoint:

az ml online-endpoint delete \
  --name <your-pf-endpoint> \
  --resource-group <your-rg> \
  --workspace-name <your-ws> \
  --yes

Delete the Prompt Flow connection:

az ml connection delete \
  --name <your-pf-connection> \
  --resource-group <your-rg> \
  --workspace-name <your-ws>

Warning

Run the cutover commands only after confirming that traffic is routed to the Agent Framework endpoint. Use --dry-run flags in your scripts to preview commands before executing them.

After cutover

Monitor Application Insights for error spikes in the first 24 hours.
Keep the archived flow YAML for at least 30 days before deleting.

Troubleshooting

No traces appearing in Application Insights

Make sure configure_azure_monitor() is called before any workflow.run() call — not after, and not inside a handler. Also verify the connection string: Azure portal > your Application Insights resource > Overview > Connection String.

`uvicorn` starts but `/ask` returns 500

The most common cause is that app.py loaded the wrong workflow file, or the target file doesn't define a module-level workflow object. Check the Application Insights trace for the full exception.

`az containerapp create` fails with an image pull error

Check:

--registry-server matches your ACR login server exactly (<name>.azurecr.io).
The Container App's managed identity (or admin credentials) has the AcrPull role on the registry.
The image tag pushed by az acr build matches the tag in --image.

Workflow hangs and never completes

The most common cause is a circular edge definition. Agent Framework uses a superstep execution model and iterates until it reaches the max_iterations limit (default: 100). Check your add_edge() calls for cycles:

WorkflowBuilder(name="MyWorkflow", max_iterations=10)

Feedback

Was this page helpful?

Last updated on 2026-04-21