Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Session affinity in model router for Microsoft Foundry keeps the turns of a conversation on the same model when that pays off. Consecutive requests that share a prompt prefix are more likely to hit that model's prompt cache, and your users get consistent behavior from turn to turn. In this article, you learn how session affinity works, use the default adaptive strategy, pin a session to one model with a session ID, turn affinity off, and check each decision.
- Send the
Foundry-Features: ModelRouterControls=V1Previewheader to turn on session affinity. With the header, the default strategy isadaptive, even if you don't sendrouting_config. - Choose a strategy in
routing_config.session_affinity.strategy:adaptive(default),sticky, ornone. - Model router identifies a conversation from a session ID you send, from the Responses
conversationID orprevious_response_id, or from a fingerprint of the earlier messages. - Affinity is best-effort and never fails a request. Model router stores only the model association, not your conversation content.
- Each response reports the strategy, the identity source, and the decision:
initialize,retain, orswitch.
Important
Session affinity in model router is in public preview and is subject to the Supplemental Terms of Use for Microsoft Azure Previews. This preview is provided without a service-level agreement, and Microsoft doesn't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities.
Prerequisites
- Auto (model router without a deployment) or a model router deployment. See Quickstart: Get started with model router.
- Python 3.10 or later with the OpenAI SDK (
pip install openai). - The
FOUNDRY_RESOURCE_ENDPOINT(https://<resource>.services.ai.azure.com) andFOUNDRY_API_KEYenvironment variables. To find the values, see Find your endpoint and key. - Familiarity with per-request controls. Session affinity settings travel in the same
routing_configobject.
How session affinity works
Without session affinity, model router routes each request independently. In a multi-turn conversation, a later turn can then go to a different model, which can't reuse the earlier model's prompt cache and might answer in a different style. Prompt-cache hits happen when the same model serves consecutive requests with overlapping prefixes, so keeping a conversation on one model raises the odds. Session affinity tells model router which requests belong together and when to keep them on the previous model.
Session affinity requires the Foundry-Features: ModelRouterControls=V1Preview header and works on all five endpoints, for Auto and for model router deployments.
Strategies
Set the strategy in routing_config.session_affinity.strategy. The older field name mode is still accepted.
| Strategy | Behavior |
|---|---|
adaptive (default) |
Keeps the conversation on the previous model while that model's prompt cache is likely still warm and its expected quality for the new turn is close to the best-ranked model for your routing mode. Otherwise, routes to the better model. Balances prompt-cache savings with quality. |
sticky |
Tries the previous model first on every turn while it's eligible. The association expires after 30 minutes without a successful request. |
none |
No affinity for this request. Model router routes it normally. |
Tip
Start with the default adaptive strategy. Use sticky when consistent behavior across turns matters more to you than picking the best-ranked model for each turn.
How model router identifies a conversation
Model router checks these sources in order, and the first match wins:
routing_config.session_affinity.session_idin the request body.- The
x-ms-session-idrequest header. - For the Responses API, the
conversationID orprevious_response_id. - Otherwise, a fingerprint of the earlier messages in the request.
A session ID has 1–256 characters and can't be only whitespace. Model router ignores an invalid session ID instead of returning an error, and moves on to the next source.
Precedence, expiry, and storage
- Policy, safety, capability, quota, and availability rules, and failover, take precedence over affinity. If the previous model isn't eligible or fails, another model serves the turn.
- Model router creates or updates a model association only after a successful request. A
stickyassociation expires after 30 minutes without a successful request. - Affinity is best-effort and never fails a request. If model router can't look up or save the association, it routes the request normally.
- Model router stores only the association between the conversation and the model. It doesn't store conversation content.
For how affinity fits into routing, see Session affinity and prompt caching.
Use adaptive affinity
With the header, model router applies the adaptive strategy to every request, even without routing_config. You don't need a session ID: model router identifies the conversation from the request itself.
Continue the conversation with previous_response_id, and send only the header:
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
# The preview header turns on adaptive session affinity. No routing_config is needed.
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
first = client.responses.create(model="auto", input="Suggest one rainy-day activity in Seattle. One sentence.")
second = client.responses.create(
model="auto",
previous_response_id=first.id, # Identifies the conversation for session affinity.
input="Add one restaurant suggestion. One sentence.",
)
for label, response in (("Turn 1", first), ("Turn 2", second)):
details = (response.model_extra or {}).get("model_selection_details") or {}
affinity = (details.get("model_router_details") or {}).get("session_affinity")
print(f"{label}: {response.model}, affinity: {affinity}")
The first turn typically reports initialize. A later turn reports retain when model router kept the previous model, or switch when a different model served the turn. To set the strategy explicitly, send "routing_config": { "session_affinity": { "strategy": "adaptive" } }.
Keep a session on one model with a session ID
Use the sticky strategy with your own session ID when you want every turn to try the previous model first. Send the ID in routing_config.session_affinity.session_id, or in the x-ms-session-id header.
Follow these rules for session IDs:
- Use 1–256 characters, and don't use only whitespace.
- Don't put secrets or personal data in a session ID. Use an opaque value, such as a random UUID.
- Use the same ID for every turn of one conversation, and a new ID for an unrelated conversation.
- If you send valid IDs in both places, the body's
session_idwins. - Model router doesn't return the session ID in the response.
import os
import uuid
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
affinity = {
"strategy": "sticky",
# Use one opaque ID per conversation. Don't put secrets or personal data in it.
"session_id": f"trip-planner-{uuid.uuid4()}",
}
routing = {"routing_config": {"session_affinity": affinity}}
messages = [{"role": "user", "content": "Suggest one rainy-day activity in Seattle. One sentence."}]
first = client.chat.completions.create(model="auto", messages=messages, extra_body=routing)
messages += [
{"role": "assistant", "content": first.choices[0].message.content},
{"role": "user", "content": "Add one restaurant suggestion. One sentence."},
]
second = client.chat.completions.create(model="auto", messages=messages, extra_body=routing)
print(f"Turn 1: {first.model}")
print(f"Turn 2: {second.model}")
Session IDs work the same way in Responses and Messages requests. A sticky association expires after 30 minutes without a successful request. After that, the next request starts a new association.
Turn off session affinity for a request
Set the strategy to none for a request that doesn't continue a conversation, or when you want the best-ranked model for a turn regardless of the previous one. Model router routes the request normally and doesn't read or update the model association.
import os
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
response = client.responses.create(
model="auto",
input="Translate 'good morning' into French.",
extra_body={"routing_config": {"session_affinity": {"strategy": "none"}}},
)
print(f"Model: {response.model}")
The response reports strategy as none and omits source and decision. To turn off affinity for all requests, don't send the Foundry-Features header. Without it, model router routes each request independently, but it also ignores per-request controls and doesn't return routing metadata.
Check the affinity decision
When you send the header, each response includes model_selection_details.model_router_details.session_affinity. The following excerpt is from the second turn of a sticky session:
{
"model": "gpt-5.6-luna-2026-07-09",
"model_selection_details": {
"model_router_details": {
"mode": "balanced",
"session_affinity": {
"strategy": "sticky",
"source": "session_id_payload",
"decision": "retain"
},
"routing_trace": [
{
"latency_ms": 840,
"attempts": [
{ "model": "gpt-5.6-luna", "result": { "status": 200 } }
]
}
]
}
}
}
| Field | What it reports |
|---|---|
strategy |
The strategy applied: adaptive, sticky, or none. |
source |
Where the conversation identity came from. Omitted when the strategy is none. |
decision |
What happened on this turn. Omitted when the strategy is none. |
source |
Meaning |
|---|---|
session_id_payload |
routing_config.session_affinity.session_id in the request body. |
session_id_header |
The x-ms-session-id header. |
derived_fingerprint |
Model router derived the identity from the request, because the request had no valid session ID. |
not_available |
No conversation identity was available for this request. |
decision |
Meaning |
|---|---|
initialize |
The first turn, or a new association. |
retain |
The same model as the previous turn served this turn. |
switch |
A different model served this turn. |
The following code sends two turns with a session ID and prints each decision:
import os
import uuid
from openai import OpenAI
client = OpenAI(
base_url=f"{os.environ['FOUNDRY_RESOURCE_ENDPOINT']}/openai/v1/",
api_key=os.environ["FOUNDRY_API_KEY"],
default_headers={"Foundry-Features": "ModelRouterControls=V1Preview"},
)
routing = {"routing_config": {"session_affinity": {"strategy": "sticky", "session_id": f"trip-{uuid.uuid4()}"}}}
def show(label, completion):
details = (completion.model_extra or {}).get("model_selection_details") or {}
affinity = (details.get("model_router_details") or {}).get("session_affinity")
if not affinity:
print(f"{label}: {completion.model} (no affinity metadata)")
return
print(f"{label}: {completion.model}, {affinity.get('strategy')}, {affinity.get('source')}, {affinity.get('decision')}")
messages = [{"role": "user", "content": "Suggest one rainy-day activity in Seattle. One sentence."}]
first = client.chat.completions.create(model="auto", messages=messages, extra_body=routing)
show("Turn 1", first)
messages += [
{"role": "assistant", "content": first.choices[0].message.content},
{"role": "user", "content": "Add one restaurant suggestion. One sentence."},
]
show("Turn 2", client.chat.completions.create(model="auto", messages=messages, extra_body=routing))
A typical run prints output similar to this example:
Turn 1: gpt-5.6-luna-2026-07-09, sticky, session_id_payload, initialize
Turn 2: gpt-5.6-luna-2026-07-09, sticky, session_id_payload, retain
- When
decisionisswitch, readrouting_traceto see which attempts failed or which model served the turn. See Interpret session affinity metadata. - If model router can't look up or save the association, it omits the whole
session_affinityobject. Don't infer a decision when the object or itsdecisionfield is missing.