Keep a conversation on one model with model router (preview)

Session affinity in model router for Microsoft Foundry keeps the turns of a conversation on the same model when that pays off. Consecutive requests that share a prompt prefix are more likely to hit that model's prompt cache, and your users get consistent behavior from turn to turn. In this article, you learn how session affinity works, use the default adaptive strategy, pin a session to one model with a session ID, turn affinity off, and check each decision.

  • Send the Foundry-Features: ModelRouterControls=V1Preview header to turn on session affinity. With the header, the default strategy is adaptive, even if you don't send routing_config.
  • Choose a strategy in routing_config.session_affinity.strategy: adaptive (default), sticky, or none.
  • Model router identifies a conversation from a session ID you send, from the Responses conversation ID or previous_response_id, or from a fingerprint of the earlier messages.
  • Affinity is best-effort and never fails a request. Model router stores only the model association, not your conversation content.
  • Each response reports the strategy, the identity source, and the decision: initialize, retain, or switch.

Important

Session affinity in model router is in public preview and is subject to the Supplemental Terms of Use for Microsoft Azure Previews. This preview is provided without a service-level agreement, and Microsoft doesn't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.

Prerequisites

  • Auto (model router without a deployment) or a model router deployment. See Quickstart: Get started with model router.
  • Python 3.10 or later with the OpenAI SDK (pip install openai).
  • The FOUNDRY_RESOURCE_ENDPOINT (https://<resource>.services.ai.azure.com) and FOUNDRY_API_KEY environment variables. To find the values, see Find your endpoint and key.
  • Familiarity with per-request controls. Session affinity settings travel in the same routing_config object.

How session affinity works

Without session affinity, model router routes each request independently. In a multi-turn conversation, a later turn can then go to a different model, which can't reuse the earlier model's prompt cache and might answer in a different style. Prompt-cache hits happen when the same model serves consecutive requests with overlapping prefixes, so keeping a conversation on one model raises the odds. Session affinity tells model router which requests belong together and when to keep them on the previous model.

Session affinity requires the Foundry-Features: ModelRouterControls=V1Preview header and works on all five endpoints, for Auto and for model router deployments.

Strategies

Set the strategy in routing_config.session_affinity.strategy. The older field name mode is still accepted.

Strategy Behavior
adaptive (default) Keeps the conversation on the previous model while that model's prompt cache is likely still warm and its expected quality for the new turn is close to the best-ranked model for your routing mode. Otherwise, routes to the better model. Balances prompt-cache savings with quality.
sticky Tries the previous model first on every turn while it's eligible. The association expires after 30 minutes without a successful request.
none No affinity for this request. Model router routes it normally.

Tip

Start with the default adaptive strategy. Use sticky when consistent behavior across turns matters more to you than picking the best-ranked model for each turn.

How model router identifies a conversation

Model router checks these sources in order, and the first match wins:

  1. routing_config.session_affinity.session_id in the request body.
  2. The x-ms-session-id request header.
  3. For the Responses API, the conversation ID or previous_response_id.
  4. Otherwise, a fingerprint of the earlier messages in the request.

A session ID has 1–256 characters and can't be only whitespace. Model router ignores an invalid session ID instead of returning an error, and moves on to the next source.

Precedence, expiry, and storage

  • Policy, safety, capability, quota, and availability rules, and failover, take precedence over affinity. If the previous model isn't eligible or fails, another model serves the turn.
  • Model router creates or updates a model association only after a successful request. A sticky association expires after 30 minutes without a successful request.
  • Affinity is best-effort and never fails a request. If model router can't look up or save the association, it routes the request normally.
  • Model router stores only the association between the conversation and the model. It doesn't store conversation content.

For how affinity fits into routing, see Session affinity and prompt caching.

Use adaptive affinity

With the header, model router applies the adaptive strategy to every request, even without routing_config. You don't need a session ID: model router identifies the conversation from the request itself.

Continue the conversation with previous_response_id, and send only the header:

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
    # The preview header turns on adaptive session affinity. No routing_config is needed.
    default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)

first = client.responses.create(model="auto", input="Suggest one rainy-day activity in Seattle. One sentence.")
second = client.responses.create(
    model="auto",
    previous_response_id=first.id,  # Identifies the conversation for session affinity.
    input="Add one restaurant suggestion. One sentence.",
)

for label, response in (("Turn 1", first), ("Turn 2", second)):
    details = (response.model_extra or {}).get("model_selection_details") or {}
    affinity = (details.get("model_router_details") or {}).get("session_affinity")
    print(f"{label}: {response.model}, affinity: {affinity}")

The first turn typically reports initialize. A later turn reports retain when model router kept the previous model, or switch when a different model served the turn. To set the strategy explicitly, send "routing_config": { "session_affinity": { "strategy": "adaptive" } }.

Keep a session on one model with a session ID

Use the sticky strategy with your own session ID when you want every turn to try the previous model first. Send the ID in routing_config.session_affinity.session_id, or in the x-ms-session-id header.

Follow these rules for session IDs:

  • Use 1–256 characters, and don't use only whitespace.
  • Don't put secrets or personal data in a session ID. Use an opaque value, such as a random UUID.
  • Use the same ID for every turn of one conversation, and a new ID for an unrelated conversation.
  • If you send valid IDs in both places, the body's session_id wins.
  • Model router doesn't return the session ID in the response.
import os
import uuid
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
    default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)

affinity = {
    "strategy": "sticky",
    # Use one opaque ID per conversation. Don't put secrets or personal data in it.
    "session_id": f"trip-planner-{uuid.uuid4()}",
}
routing = {"routing_config": {"session_affinity": affinity}}

messages = [{"role": "user", "content": "Suggest one rainy-day activity in Seattle. One sentence."}]
first = client.chat.completions.create(model="auto", messages=messages, extra_body=routing)
messages += [
    {"role": "assistant", "content": first.choices[0].message.content},
    {"role": "user", "content": "Add one restaurant suggestion. One sentence."},
]
second = client.chat.completions.create(model="auto", messages=messages, extra_body=routing)

print(f"Turn 1: {first.model}")
print(f"Turn 2: {second.model}")

Session IDs work the same way in Responses and Messages requests. A sticky association expires after 30 minutes without a successful request. After that, the next request starts a new association.

Turn off session affinity for a request

Set the strategy to none for a request that doesn't continue a conversation, or when you want the best-ranked model for a turn regardless of the previous one. Model router routes the request normally and doesn't read or update the model association.

import os
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
    default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)

response = client.responses.create(
    model="auto",
    input="Translate 'good morning' into French.",
    extra_body={"routing_config": {"session_affinity": {"strategy": "none"}}},
)
print(f"Model: {response.model}")

The response reports strategy as none and omits source and decision. To turn off affinity for all requests, don't send the Foundry-Features header. Without it, model router routes each request independently, but it also ignores per-request controls and doesn't return routing metadata.

Check the affinity decision

When you send the header, each response includes model_selection_details.model_router_details.session_affinity. The following excerpt is from the second turn of a sticky session:

{
  "model": "gpt-5.6-luna-2026-07-09",
  "model_selection_details": {
    "model_router_details": {
      "mode": "balanced",
      "session_affinity": {
        "strategy": "sticky",
        "source": "session_id_payload",
        "decision": "retain"
      },
      "routing_trace": [
        {
          "latency_ms": 840,
          "attempts": [
            { "model": "gpt-5.6-luna", "result": { "status": 200 } }
          ]
        }
      ]
    }
  }
}
Field What it reports
strategy The strategy applied: adaptive, sticky, or none.
source Where the conversation identity came from. Omitted when the strategy is none.
decision What happened on this turn. Omitted when the strategy is none.
source Meaning
session_id_payload routing_config.session_affinity.session_id in the request body.
session_id_header The x-ms-session-id header.
derived_fingerprint Model router derived the identity from the request, because the request had no valid session ID.
not_available No conversation identity was available for this request.
decision Meaning
initialize The first turn, or a new association.
retain The same model as the previous turn served this turn.
switch A different model served this turn.

The following code sends two turns with a session ID and prints each decision:

import os
import uuid
from openai import OpenAI

client = OpenAI(
    base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
    api_key=os.environ["FOUNDRY_API_KEY"],
    default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
routing = {"routing_config": {"session_affinity": {"strategy": "sticky", "session_id": f"trip-{uuid.uuid4()}"}}}


def show(label, completion):
    details = (completion.model_extra or {}).get("model_selection_details") or {}
    affinity = (details.get("model_router_details") or {}).get("session_affinity")
    if not affinity:
        print(f"{label}: {completion.model} (no affinity metadata)")
        return
    print(f"{label}: {completion.model}, {affinity.get('strategy')}, {affinity.get('source')}, {affinity.get('decision')}")


messages = [{"role": "user", "content": "Suggest one rainy-day activity in Seattle. One sentence."}]
first = client.chat.completions.create(model="auto", messages=messages, extra_body=routing)
show("Turn 1", first)
messages += [
    {"role": "assistant", "content": first.choices[0].message.content},
    {"role": "user", "content": "Add one restaurant suggestion. One sentence."},
]
show("Turn 2", client.chat.completions.create(model="auto", messages=messages, extra_body=routing))

A typical run prints output similar to this example:

Turn 1: gpt-5.6-luna-2026-07-09, sticky, session_id_payload, initialize
Turn 2: gpt-5.6-luna-2026-07-09, sticky, session_id_payload, retain
  • When decision is switch, read routing_trace to see which attempts failed or which model served the turn. See Interpret session affinity metadata.
  • If model router can't look up or save the association, it omits the whole session_affinity object. Don't infer a decision when the object or its decision field is missing.