Reduce startup latency for hosted agents

When a new or resumed session starts, a hosted agent in Microsoft Foundry waits for its container to start and report ready before it can process requests. This article shows how to reduce that container readiness time by moving work into the image build and keeping startup small.

To reuse initialized application state across sessions instead of repeating startup work, also see Enable memory snapshots for hosted agents.

Prerequisites

  • A hosted agent deployed from a container image. To deploy one, see Deploy a hosted agent.
  • Docker, to build the image.

Measure container readiness

Before you optimize, confirm that startup time is the source of the delay. Send a request with the x-ms-debug-latency-enabled: true header, and compare x-ms-debug-latency-container-readiness-ms across several cold or resumed requests. For steps, see Diagnose invocation latency.

If readiness time is low but requests are still slow, the delay is in request handling. See Investigate agent and model latency.

Reduce container readiness time

The container readiness value includes the time required to start your process, load the application, and return HTTP 200 from /readiness. Add timestamps to startup logs to identify slow imports, dependency loading, network calls, and other initialization work.

Apply these optimizations to the slow steps you identify:

  • Install dependencies and compile code when you build the image. Don't run package installation, restore, or compilation at container startup.
  • Use a multistage build and exclude build tools, package caches, tests, and other development files from the runtime image.
  • Avoid model calls, migrations, tool discovery, and asset downloads before readiness. Parallelize independent initialization that must finish first.
  • Keep /readiness focused on whether the agent can accept requests. If you defer initialization, measure the first request to ensure you didn't move the startup delay into request handling.
  • Enable memory snapshots so new sessions restore initialized state instead of repeating startup work. See Enable memory snapshots for hosted agents.

Install dependencies in a virtual environment that you copy into the runtime image. Compile dependencies on a best-effort basis so an unparsable file in a third-party package doesn't fail the build. Compile the application strictly so syntax errors fail the build.

Set PYTHONNOUSERSITE=1 in each Python image stage before starting the interpreter. This setting disables user-site package loading without affecting the virtual environment in the following Dockerfile. Also configure the agent's deployment environment as described in Configure Python startup.

FROM python:3.13-slim AS build

ENV PYTHONNOUSERSITE=1
WORKDIR /app
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .
RUN PYTHONDONTWRITEBYTECODE= python -m compileall -q /opt/venv \
      || true; \
    PYTHONDONTWRITEBYTECODE= python -m compileall -q /app

FROM python:3.13-slim AS final

ENV PYTHONNOUSERSITE=1
ENV PATH="/opt/venv/bin:$PATH"
WORKDIR /app

COPY --from=build /opt/venv /opt/venv
COPY --from=build /app /app

# Precompile the standard library in the runtime image.
RUN PYTHONDONTWRITEBYTECODE= python -m compileall -q \
    "$(python -c "import sysconfig; print(sysconfig.get_path('stdlib'))")" \
    || true

CMD ["python", "main.py"]

Clearing PYTHONDONTWRITEBYTECODE for this build step permits Python to write the bytecode into the image. The environment variable blocks bytecode writes, not reads, so the running container can still use the compiled files.

Reference: compileall - Byte-compile Python libraries

Build either image for the x86-64 architecture required by hosted agents:

docker build --platform linux/amd64 -t my-agent:latest .

Here, linux/amd64 is a Docker platform, not a .NET runtime identifier.

Reference: Hosted agent container requirements

Related content

  • Last updated on