Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
System tools let a voice agent in Microsoft Foundry transfer calls, collect keypad input, send messages, and end conversations. Add the tools to your agent definition and describe when the agent should use them.
This article uses a customer-support call to show how system tools work together. The agent collects a verification code, looks up a support case through your Model Context Protocol (MCP) server, and transfers the caller to a person when needed.
Use Choose a configuration method at the top of this article to select Python SDK or REST API.
Voice agents are generally available, with some features still in preview. See General availability and preview features.
Prerequisites
- A Foundry project and permission to create or update agents.
- Access to a voice model in your project. See Configure a voice agent for model selection and agent settings.
Choose the tools
Choose tools based on the action you want the agent to perform:
| Tool name | Use it to |
|---|---|
end_conversation |
End the conversation after the agent's farewell plays. |
language_detection |
Adjust the conversation language when the caller requests or starts using another language. |
digit_recognition |
Retrieve the transcript of the caller's most recent spoken response, such as an account number. |
transfer |
Transfer a phone call to a configured destination group. |
collect_sms |
Collect numeric input from an SMS sent by a specified sender. |
collect_dtmf |
Collect digits entered on the phone keypad. |
send_dtmf |
Send keypad tones, such as a menu selection, during a call. |
send_sms |
Send a text message during a call. |
Caution
transfer, collect_sms, collect_dtmf, send_dtmf, and send_sms act on phone calls. Telephony channels aren't currently supported for voice agents in network-isolated projects, such as projects that use virtual network injection or a Foundry resource with public network access disabled, so these tools aren't available there.
Add the tools to the agent's tools array. Each tool accepts an optional description that explains when the agent should use it.
For spoken identifiers, digit_recognition gives the agent a transcript to use when reading back and confirming a number. For a private keypad code, use collect_dtmf with secret: true.
Choose how to control a call
Call control applies to active inbound and outbound calls. Your application can request an action through an API, or the agent can choose an action through a system tool.
| Task | Application control | Agent control |
|---|---|---|
| Transfer an active call | Send a transfer request with the call ID and a configured transfer target name. | The transfer tool selects a target from the agent definition. |
| End an active call | Send an end-call request with the call ID. | The end_conversation tool ends the conversation after the farewell plays. |
For application-controlled actions, use the call ID returned by the Calls API.
Enable a tool
Apply these steps to an existing agent, or follow the complete inbound support example.
Add each capability to the agent's
toolsarray withtype: systemand its built-inname. Include every tool you want to keep when replacing the array.Set
tool_choicetoautoand describe when to use the tools in the agent's instructions. The model supplies the arguments during the conversation.For example, merge these settings into your agent definition to end a conversation after a farewell. Keep your existing instructions and other tools when adding this behavior:
{ "instructions": "When the caller's request is complete, say goodbye and call end_conversation in the same response.", "tool_choice": "auto", "tools": [ { "type": "system", "name": "end_conversation" } ] }Save the updated definition and start a new session with that version.
In this example, the service ends the conversation after the agent's farewell finishes playing.
Handle an inbound support call
The support agent asks for a customer ID, collects a verification code through the phone keypad, and retrieves the customer's case summary. It offers a transfer when the caller needs a person.
For this example, prepare:
- An Azure Communication Services (ACS) project connection and a number configured for Teams Phone extensibility.
- A phone to place a test call and a Teams user or resource account to receive a transfer.
- Your own MCP server with verification and case-lookup tools. The following steps connect these tools to the agent.
Prepare the business tools
For this example, your support application exposes two custom tools through an MCP server. The following sample shows the tool definitions in the server's tools/list result. Adapt the names and schemas to your application's verification and case-lookup operations.
{
"tools": [
{
"name": "verify_customer",
"description": "Verify a customer ID and code. Return a verification token on success.",
"inputSchema": {
"type": "object",
"properties": {
"customer_id": { "type": "string" },
"code": { "type": "string" }
},
"required": ["customer_id", "code"]
}
},
{
"name": "get_case_status",
"description": "Use a verification token to retrieve the customer's support case summary.",
"inputSchema": {
"type": "object",
"properties": {
"verification_token": { "type": "string" }
},
"required": ["verification_token"]
}
}
]
}
Implement verify_customer to check the code and return a verification_token on success. Implement get_case_status to check that token before returning the customer's case summary. Return the verification result to the agent while keeping the code private.
- Create a key-based MCP project connection for your server. Store the headers it expects, such as
Authorization: Bearer <server-token>or an API-key header, and save the connection ID. - Confirm that your test customer can obtain a code through your existing verification process. The agent collects that code during the call.
Create the support agent
The agent combines your business tools with collect_dtmf, transfer, and end_conversation. Choose Python or REST to create it.
To save transcripts and audio for later review, enable conversation storage by setting store: true in the agent definition. See Voice agent observability for access and retention settings.
Complete the Python setup. Set
FOUNDRY_PROJECT_ENDPOINTto your project endpoint.Set
MCP_SERVER_URL,MCP_PROJECT_CONNECTION_ID, andSUPPORT_TEAMS_OBJECT_IDto your MCP server URL, project connection ID, and the transfer recipient's Teams object GUID.Create
contoso-inbound-support. Use your project's model and voice settings:import os from datetime import timedelta from azure.ai.projects import AIProjectClient from azure.ai.projects.models import ( VoiceAgentAudioConfig, VoiceAgentAudioOutputConfig, VoiceAgentCollectDtmfSystemTool, VoiceAgentDefinition, VoiceAgentEndConversationSystemTool, VoiceAgentMcpTool, VoiceAgentTemplateGreetingConfig, VoiceAgentTransferDestination, VoiceAgentTransferRouting, VoiceAgentTransferSystemTool, VoiceAgentTransferTarget, ) from azure.identity import DefaultAzureCredential agent_name = "contoso-inbound-support" instructions = """ Help callers check their support cases. Ask for the customer ID. Start collect_dtmf with secret=true, output_variable_name=verification_code, and completion_mode={"type":"fixed_length","length":6}. Prompt for the six-digit code after collection reports listening. After collection completes, call verify_customer with customer_id and code. Use the actual output_ref name wrapped in double braces for code; build that reference at call time. If collection times out or fails, offer a retry or a support transfer. When verification succeeds, pass its verification_token to get_case_status. Explain the returned case summary in a short spoken response. If verification fails or the caller asks for a person, announce a transfer and use transfer with target=support and a short reason. If transfer reports transfer_not_answered, offer to continue helping or retry at the caller's request. For an uncertain transfer result, explain that the transfer status needs checking. Request another transfer only after its outcome is confirmed. When the caller confirms the request is complete, say goodbye and use end_conversation in the same response. """.strip() definition = VoiceAgentDefinition( model_type="managed", model="gpt-4.1", instructions=instructions, greeting=VoiceAgentTemplateGreetingConfig( text=( "Welcome to Contoso support. I'm a virtual assistant. " "I can check your support case or connect you with a " "representative. How can I help?" ), ), audio=VoiceAgentAudioConfig( output=VoiceAgentAudioOutputConfig( voice_type="azure-standard", voice="en-US-AvaMultilingualNeural", ), ), tool_choice="auto", tools=[ VoiceAgentMcpTool( server_label="contoso_support", server_url=os.environ["MCP_SERVER_URL"], project_connection_id=os.environ["MCP_PROJECT_CONNECTION_ID"], allowed_tools=["verify_customer", "get_case_status"], require_approval="never", ), VoiceAgentEndConversationSystemTool(), VoiceAgentCollectDtmfSystemTool( timeout=timedelta(seconds=90), ), VoiceAgentTransferSystemTool( timeout=timedelta(seconds=60), targets=[ VoiceAgentTransferTarget( name="support", description="A support representative for case questions.", destinations=[ VoiceAgentTransferDestination( type="teams", value=os.environ["SUPPORT_TEAMS_OBJECT_ID"], ), ], routing=VoiceAgentTransferRouting( strategy="sequential", ring_timeout=timedelta(seconds=20), ), ), ], ), ], ) with ( DefaultAzureCredential() as credential, AIProjectClient( endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], credential=credential, ) as project_client, ): created = project_client.agents.create_version( agent_name=agent_name, definition=definition, ) print(f"Created {agent_name}, version {created.version}")Reference: AIProjectClient | DefaultAzureCredential.
The system-tool classes supply their
typeandnamevalues. Usetimedeltafor the timeout settings.Save the returned agent name and version. Test the greeting in the playground before connecting the support line.
Complete the REST setup to get your project endpoint and access token. Replace
{projectEndpoint}and<access-token>in the request with those values.Create
contoso-inbound-support. Replace the model, voice, MCP URL, connection ID, and the support recipient's Teams object GUID with your settings.POST {projectEndpoint}/agents?api-version=v1 Authorization: Bearer <access-token> Content-Type: application/json { "name": "contoso-inbound-support", "definition": { "kind": "voice", "sub_protocol": "realtime", "model_type": "managed", "model": "gpt-4.1", "instructions": "Help callers check their support cases. Ask for the customer ID.\nStart collect_dtmf with secret=true, output_variable_name=verification_code,\nand completion_mode={\"type\":\"fixed_length\",\"length\":6}.\nPrompt for the six-digit code after collection reports listening.\nAfter collection completes, call verify_customer with customer_id and code. Use the actual output_ref name wrapped in double braces for code; build that reference at call time.\nIf collection times out or fails, offer a retry or a support transfer.\nWhen verification succeeds, pass its verification_token to get_case_status.\nExplain the returned case summary in a short spoken response.\nIf verification fails or the caller asks for a person, announce a transfer\nand use transfer with target=support and a short reason.\nIf transfer reports transfer_not_answered, offer to continue helping or retry at the caller's request.\nFor an uncertain transfer result, explain that the transfer status needs checking. Request another transfer only after its outcome is confirmed.\nWhen the caller confirms the request is complete, say goodbye and use\nend_conversation in the same response.", "greeting": { "type": "template", "text": "Welcome to Contoso support. I'm a virtual assistant. I can check your support case or connect you with a representative. How can I help?" }, "audio": { "output": { "voice_type": "azure-standard", "voice": "en-US-AvaMultilingualNeural" } }, "tool_choice": "auto", "tools": [ { "type": "mcp", "server_label": "contoso_support", "server_url": "https://support.contoso.com/mcp", "project_connection_id": "<mcp-project-connection-id>", "allowed_tools": ["verify_customer", "get_case_status"], "require_approval": "never" }, { "type": "system", "name": "end_conversation" }, { "type": "system", "name": "collect_dtmf", "timeout_seconds": 90 }, { "type": "system", "name": "transfer", "timeout_seconds": 60, "targets": [ { "name": "support", "description": "A support representative for case questions.", "destinations": [ { "type": "teams", "value": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb" } ], "routing": { "strategy": "sequential", "ring_timeout_seconds": 20 } } ] } ] } }Save the returned agent name and version. Test the greeting in the playground before connecting the support line.
allowed_tools exposes the two business tools, and require_approval: never lets the agent invoke them during the call. Your MCP server verifies the customer before returning case information.
Connect and test the support line
Create the inbound binding, register its incoming-call URL, and test the support workflow.
Create the inbound binding
Route the Teams Phone extensibility number through your ACS connection to the saved agent.
Set
ACS_CONNECTION_NAMEandRESOURCE_ACCOUNT_OBJECT_IDto the ACS connection name and the Teams resource-account GUID for your support number.Create the inbound binding. Replace the display phone number with your support number:
import os from azure.ai.projects import AIProjectClient from azure.ai.projects.models import TeamsPhoneExtensibilityTelephonyBinding from azure.identity import DefaultAzureCredential with ( DefaultAzureCredential() as credential, AIProjectClient( endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], credential=credential, ) as project_client, ): binding = project_client.voice_agents.telephony.create_binding( agent_name="contoso-inbound-support", binding=TeamsPhoneExtensibilityTelephonyBinding( connection_name=os.environ["ACS_CONNECTION_NAME"], resource_account_object_id=os.environ["RESOURCE_ACCOUNT_OBJECT_ID"], phone_number="+14255550100", label="Contoso support", ), ) print(f"Binding ID: {binding.id}") print(f"Incoming call URL: {binding.incoming_call_url}")Reference: AIProjectClient | DefaultAzureCredential.
Create the inbound binding. Replace the resource-account GUID, ACS connection name, and display phone number with your values:
POST {projectEndpoint}/agents/contoso-inbound-support/telephony/bindings?api-version=v1 Authorization: Bearer <access-token> Content-Type: application/json { "provider": "teams_phone_extensibility", "connection_name": "contoso-acs", "resource_account_object_id": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "phone_number": "+14255550100", "label": "Contoso support" }
Register the incoming-call URL and test the line
After you create the binding, complete these steps for either Python or REST:
- Save the binding ID and
incoming_call_urlfrom the response. Use that URL to register the Event Grid subscription. - Call the support number from your test phone. Follow the greeting, provide your test customer ID, and enter the code after the agent prompts you.
- Check both outcomes: a verified case lookup, and a request to speak with a person. Complete a separate call to test the farewell and automatic ending.
Use this sequence to verify the customer experience:
| Stage | What happens |
|---|---|
| Greeting | The agent introduces itself and asks how it can help. |
| Collection | The agent starts collect_dtmf, waits for listening, and asks the caller to enter the code. |
| Verification | Private collection returns an output_ref. The agent passes the customer ID and a reference such as {{verification_code}} to verify_customer. |
| Case lookup | The agent passes the returned verification token to get_case_status and explains its authorized summary. |
| Handoff or completion | The agent uses transfer when a person is needed, or speaks a farewell and uses end_conversation. |
Use the returned output_ref when passing the code to your business tool. See Use a private result.
Configure a transfer target
The example defines a target named support in the transfer tool. To adapt this ACS example for your support team:
- Set a unique
nameand adescriptionthat explains when to use the target. - Add up to 16 destination addresses. For a Teams destination, set
typetoteamsandvalueto the Teams user or resource-account object ID. For a phone-number destination, usetype: phone_numberand an E.164 number. Phone-number transfers also require an E.164 caller ID for the call. - Choose
routing.strategy:sequentialtries destinations in order, andrandomshuffles them for each transfer. - Set
routing.ring_timeout_secondsto control how long each destination rings. It defaults to 20 seconds; ACS supports up to 180 seconds. Keep it within the transfer tool'stimeout_seconds.
In the agent's instructions, describe when to transfer and what to say before the handoff. If the result is transfer_not_answered, offer to continue helping or try another target. For an uncertain result, have your application check the call record and provider status before requesting another transfer.
For application-controlled transfers, configure API transfer targets and manage the active call by its call ID.
Collect keypad or SMS input
Use collect_dtmf to collect keypad input on Twilio or ACS calls. For collect_sms, first configure incoming SMS for a number bound to the agent.
Set the collection tool's timeout_seconds to the time you want to allow for input. The default is 60 seconds, and the supported range is 1–300 seconds. The example uses 90 seconds.
In the agent's instructions, use secret: true and an output_variable_name for a verification code or other private value. Successful collection returns a reference that the agent can pass to your business tool. Use secret: false when the agent should read the collected value directly.
Tell the agent which keypad completion mode to use:
completion_mode |
Completion rule |
|---|---|
{"type":"fixed_length","length":6} |
Exactly six digits arrive. length defaults to 6 and supports 1–64. |
{"type":"terminator","terminator":"#"} |
The caller enters digits followed by #. The returned value contains the digits before the terminator. * is also supported. |
{"type":"inter_digit_silence","inter_digit_timeout_seconds":2} |
The caller enters at least one digit, then pauses for two seconds. |
The first-digit wait defaults to 15 seconds, and the pause between digits defaults to 5 seconds. To adjust them, specify first_digit_timeout_seconds in the collection arguments and inter_digit_timeout_seconds inside completion_mode. Collected strings preserve leading zeros.
For private SMS collection, instruct the agent to use the caller's SMS number as from_phone_number. For example, the tool arguments can be:
{
"secret": true,
"from_phone_number": "+14255550123",
"output_variable_name": "verification_code"
}
Use the sender's E.164 number in from_phone_number. The tool collects one numeric message from that number, and your business service verifies the code.
Instruct the agent to ask for input after collection reports listening, then wait for the result. Offer a retry or support transfer if collection expires or fails.
Configure incoming SMS
To collect a numeric message during a call, route incoming SMS from your Twilio number to the agent:
In the Twilio number's Messaging settings, set A message comes in to Webhook and select HTTP POST. Use this URL:
{projectEndpoint}/agents/{agentName}/endpoint/protocols/voice/telephony/twilio/callbacks/collection-sms?api-version=v1&phone={encodedAgentPhoneNumber}Use your Foundry project endpoint and agent name. Set
phoneto the URL-encoded bound number; for example,+14255550100becomes%2B14255550100.If the number belongs to a Twilio Messaging Service, configure that service to use the phone number's incoming-message webhook. Save the settings.
During the call, the agent starts collect_sms with the sender's number in from_phone_number. After the tool reports listening, ask the caller to send their numeric message to the bound agent number.
Use a private result
A successful private collection returns a name in output_ref. Wrap that returned name in double braces when passing it to an MCP tool, send_sms, or send_dtmf. For example, if output_ref is verification_code, pass:
{
"code": "{{verification_code}}"
}
The service supplies the private value to the receiving tool while the agent sees the reference. Have your business tool return a verification result and keep the private value out of its response and logs.
Send tones or an SMS
Enable send_dtmf to navigate a telephone menu or send_sms to send a text during a call. Describe when to use the tool and what to send in the agent's instructions.
| Tool | Arguments to describe in the instructions |
|---|---|
send_dtmf |
digits, such as 1# for a telephone menu selection. Use 0–9, *, and #. |
send_sms |
to_phone_number, the recipient's E.164 number, and message, the text to send. |
For send_dtmf, send 1–18 tones per invocation on ACS or 1–64 on Twilio and SIP. To send a private value, use its returned reference, such as {{verification_code}}#.
For a Twilio call, bind an SMS-capable number to the agent. The send_sms tool sends from that bound number. Set timeout_seconds in either tool's configuration to control how long the action can take.
Observe execution and test
Use separate test calls for the following scenarios. Review call activity in call records and dialogue in stored conversations.
- Complete a successful verification and case lookup. Confirm that the caller hears the correct summary and your MCP service receives the verification input.
- Test an incorrect code and a collection timeout. Confirm that the agent offers a retry or a support transfer.
- Test a transfer to the support recipient and a call where that recipient is unavailable. Check that the agent responds appropriately to each outcome.
- Complete the caller's request and confirm that the farewell finishes before the call ends.
If your application reads voice protocol events, use input_collection.status to inspect collection progress and phone-action feedback. The event's call_id identifies the tool invocation. Use the telephony call ID for Calls API requests. A system_tool item's completed status describes argument generation. Use the execution feedback and call record to check the action's result.
| Outcome | Meaning and next step |
|---|---|
provider_accepted |
The provider accepted the request. Confirm SMS delivery or the telephone menu's response through the provider or conversation flow. |
transfer_not_answered |
The transfer target didn't answer, and the original conversation remains available. Offer to continue helping or retry at the caller's request. |
provider_not_executed |
The service stopped before sending or dialing. Review the cause before starting a new request. |
provider_failed_or_unsupported |
The action failed validation or execution. Check the tool arguments and provider configuration. |
provider_result_uncertain |
Check the call record and provider status to establish the outcome before requesting the action again. |
After testing, remove temporary bindings that you no longer need.
Troubleshoot tool availability
Use the symptom to choose what to check:
| Symptom | What to check |
|---|---|
| The agent skips a tool | Confirm that the saved tools array includes the tool, tool_choice is auto, and the instructions describe when to use the tool. Start a new session with the updated version. |
| Keypad collection times out | Check that the agent waits for listening before prompting and that the caller follows the configured completion mode. |
| A transfer fails | Check the target's name and destination, the configured caller ID, and whether the recipient can receive the call. |
| SMS sending fails | Check the sender number's SMS capability, its binding to the agent, and the provider result. |
For call-query examples and an investigation workflow, see Troubleshoot telephony.
Related content
Continue with these guides to configure the agent or create outbound calling workflows: