Configure system tools for a voice agent

System tools let a voice agent in Microsoft Foundry transfer calls, collect keypad input, send messages, and end conversations. Add the tools to your agent definition and describe when the agent should use them.

This article uses a customer-support call to show how system tools work together. The agent collects a verification code, looks up a support case through your Model Context Protocol (MCP) server, and transfers the caller to a person when needed.

Use Choose a configuration method at the top of this article to select Python SDK or REST API.

Voice agents are generally available, with some features still in preview. See General availability and preview features.

Prerequisites

Choose the tools

Choose tools based on the action you want the agent to perform:

Tool name Use it to
end_conversation End the conversation after the agent's farewell plays.
language_detection Adjust the conversation language when the caller requests or starts using another language.
digit_recognition Retrieve the transcript of the caller's most recent spoken response, such as an account number.
transfer Transfer a phone call to a configured destination group.
collect_sms Collect numeric input from an SMS sent by a specified sender.
collect_dtmf Collect digits entered on the phone keypad.
send_dtmf Send keypad tones, such as a menu selection, during a call.
send_sms Send a text message during a call.

Caution

transfer, collect_sms, collect_dtmf, send_dtmf, and send_sms act on phone calls. Telephony channels aren't currently supported for voice agents in network-isolated projects, such as projects that use virtual network injection or a Foundry resource with public network access disabled, so these tools aren't available there.

Add the tools to the agent's tools array. Each tool accepts an optional description that explains when the agent should use it.

For spoken identifiers, digit_recognition gives the agent a transcript to use when reading back and confirming a number. For a private keypad code, use collect_dtmf with secret: true.

Choose how to control a call

Call control applies to active inbound and outbound calls. Your application can request an action through an API, or the agent can choose an action through a system tool.

Task Application control Agent control
Transfer an active call Send a transfer request with the call ID and a configured transfer target name. The transfer tool selects a target from the agent definition.
End an active call Send an end-call request with the call ID. The end_conversation tool ends the conversation after the farewell plays.

For application-controlled actions, use the call ID returned by the Calls API.

Enable a tool

Apply these steps to an existing agent, or follow the complete inbound support example.

  1. Add each capability to the agent's tools array with type: system and its built-in name. Include every tool you want to keep when replacing the array.

  2. Set tool_choice to auto and describe when to use the tools in the agent's instructions. The model supplies the arguments during the conversation.

    For example, merge these settings into your agent definition to end a conversation after a farewell. Keep your existing instructions and other tools when adding this behavior:

    {
      "instructions": "When the caller's request is complete, say goodbye and call end_conversation in the same response.",
      "tool_choice": "auto",
      "tools": [
        { "type": "system", "name": "end_conversation" }
      ]
    }
    
  3. Save the updated definition and start a new session with that version.

In this example, the service ends the conversation after the agent's farewell finishes playing.

Handle an inbound support call

The support agent asks for a customer ID, collects a verification code through the phone keypad, and retrieves the customer's case summary. It offers a transfer when the caller needs a person.

For this example, prepare:

  • An Azure Communication Services (ACS) project connection and a number configured for Teams Phone extensibility.
  • A phone to place a test call and a Teams user or resource account to receive a transfer.
  • Your own MCP server with verification and case-lookup tools. The following steps connect these tools to the agent.

Prepare the business tools

For this example, your support application exposes two custom tools through an MCP server. The following sample shows the tool definitions in the server's tools/list result. Adapt the names and schemas to your application's verification and case-lookup operations.

{
  "tools": [
    {
      "name": "verify_customer",
      "description": "Verify a customer ID and code. Return a verification token on success.",
      "inputSchema": {
        "type": "object",
        "properties": {
          "customer_id": { "type": "string" },
          "code": { "type": "string" }
        },
        "required": ["customer_id", "code"]
      }
    },
    {
      "name": "get_case_status",
      "description": "Use a verification token to retrieve the customer's support case summary.",
      "inputSchema": {
        "type": "object",
        "properties": {
          "verification_token": { "type": "string" }
        },
        "required": ["verification_token"]
      }
    }
  ]
}

Implement verify_customer to check the code and return a verification_token on success. Implement get_case_status to check that token before returning the customer's case summary. Return the verification result to the agent while keeping the code private.

  1. Create a key-based MCP project connection for your server. Store the headers it expects, such as Authorization: Bearer <server-token> or an API-key header, and save the connection ID.
  2. Confirm that your test customer can obtain a code through your existing verification process. The agent collects that code during the call.

Create the support agent

The agent combines your business tools with collect_dtmf, transfer, and end_conversation. Choose Python or REST to create it.

To save transcripts and audio for later review, enable conversation storage by setting store: true in the agent definition. See Voice agent observability for access and retention settings.

  1. Complete the Python setup. Set FOUNDRY_PROJECT_ENDPOINT to your project endpoint.

  2. Set MCP_SERVER_URL, MCP_PROJECT_CONNECTION_ID, and SUPPORT_TEAMS_OBJECT_ID to your MCP server URL, project connection ID, and the transfer recipient's Teams object GUID.

  3. Create contoso-inbound-support. Use your project's model and voice settings:

    import os
    from datetime import timedelta
    
    from azure.ai.projects import AIProjectClient
    from azure.ai.projects.models import (
        VoiceAgentAudioConfig,
        VoiceAgentAudioOutputConfig,
        VoiceAgentCollectDtmfSystemTool,
        VoiceAgentDefinition,
        VoiceAgentEndConversationSystemTool,
        VoiceAgentMcpTool,
        VoiceAgentTemplateGreetingConfig,
        VoiceAgentTransferDestination,
        VoiceAgentTransferRouting,
        VoiceAgentTransferSystemTool,
        VoiceAgentTransferTarget,
    )
    from azure.identity import DefaultAzureCredential
    
    agent_name = "contoso-inbound-support"
    instructions = """
    Help callers check their support cases. Ask for the customer ID.
    Start collect_dtmf with secret=true, output_variable_name=verification_code,
    and completion_mode={"type":"fixed_length","length":6}.
    Prompt for the six-digit code after collection reports listening.
    After collection completes, call verify_customer with customer_id and code.
    Use the actual output_ref name wrapped in double braces for code;
    build that reference at call time.
    If collection times out or fails, offer a retry or a support transfer.
    When verification succeeds, pass its verification_token to get_case_status.
    Explain the returned case summary in a short spoken response.
    If verification fails or the caller asks for a person, announce a transfer
    and use transfer with target=support and a short reason.
    If transfer reports transfer_not_answered, offer to continue helping or retry
    at the caller's request.
    For an uncertain transfer result, explain that the transfer status needs
    checking. Request another transfer only after its outcome is confirmed.
    When the caller confirms the request is complete, say goodbye and use
    end_conversation in the same response.
    """.strip()
    
    definition = VoiceAgentDefinition(
        model_type="managed",
        model="gpt-4.1",
        instructions=instructions,
        greeting=VoiceAgentTemplateGreetingConfig(
            text=(
                "Welcome to Contoso support. I'm a virtual assistant. "
                "I can check your support case or connect you with a "
                "representative. How can I help?"
            ),
        ),
        audio=VoiceAgentAudioConfig(
            output=VoiceAgentAudioOutputConfig(
                voice_type="azure-standard",
                voice="en-US-AvaMultilingualNeural",
            ),
        ),
        tool_choice="auto",
        tools=[
            VoiceAgentMcpTool(
                server_label="contoso_support",
                server_url=os.environ["MCP_SERVER_URL"],
                project_connection_id=os.environ["MCP_PROJECT_CONNECTION_ID"],
                allowed_tools=["verify_customer", "get_case_status"],
                require_approval="never",
            ),
            VoiceAgentEndConversationSystemTool(),
            VoiceAgentCollectDtmfSystemTool(
                timeout=timedelta(seconds=90),
            ),
            VoiceAgentTransferSystemTool(
                timeout=timedelta(seconds=60),
                targets=[
                    VoiceAgentTransferTarget(
                        name="support",
                        description="A support representative for case questions.",
                        destinations=[
                            VoiceAgentTransferDestination(
                                type="teams",
                                value=os.environ["SUPPORT_TEAMS_OBJECT_ID"],
                            ),
                        ],
                        routing=VoiceAgentTransferRouting(
                            strategy="sequential",
                            ring_timeout=timedelta(seconds=20),
                        ),
                    ),
                ],
            ),
        ],
    )
    
    with (
        DefaultAzureCredential() as credential,
        AIProjectClient(
            endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
            credential=credential,
        ) as project_client,
    ):
        created = project_client.agents.create_version(
            agent_name=agent_name,
            definition=definition,
        )
        print(f"Created {agent_name}, version {created.version}")
    

    Reference: AIProjectClient | DefaultAzureCredential.

    The system-tool classes supply their type and name values. Use timedelta for the timeout settings.

  4. Save the returned agent name and version. Test the greeting in the playground before connecting the support line.

  1. Complete the REST setup to get your project endpoint and access token. Replace {projectEndpoint} and <access-token> in the request with those values.

  2. Create contoso-inbound-support. Replace the model, voice, MCP URL, connection ID, and the support recipient's Teams object GUID with your settings.

    POST {projectEndpoint}/agents?api-version=v1
    Authorization: Bearer <access-token>
    Content-Type: application/json
    
    {
      "name": "contoso-inbound-support",
      "definition": {
        "kind": "voice",
        "sub_protocol": "realtime",
        "model_type": "managed",
        "model": "gpt-4.1",
        "instructions": "Help callers check their support cases. Ask for the customer ID.\nStart collect_dtmf with secret=true, output_variable_name=verification_code,\nand completion_mode={\"type\":\"fixed_length\",\"length\":6}.\nPrompt for the six-digit code after collection reports listening.\nAfter collection completes, call verify_customer with customer_id and code. Use the actual output_ref name wrapped in double braces for code; build that reference at call time.\nIf collection times out or fails, offer a retry or a support transfer.\nWhen verification succeeds, pass its verification_token to get_case_status.\nExplain the returned case summary in a short spoken response.\nIf verification fails or the caller asks for a person, announce a transfer\nand use transfer with target=support and a short reason.\nIf transfer reports transfer_not_answered, offer to continue helping or retry at the caller's request.\nFor an uncertain transfer result, explain that the transfer status needs checking. Request another transfer only after its outcome is confirmed.\nWhen the caller confirms the request is complete, say goodbye and use\nend_conversation in the same response.",
        "greeting": {
          "type": "template",
          "text": "Welcome to Contoso support. I'm a virtual assistant. I can check your support case or connect you with a representative. How can I help?"
        },
        "audio": {
          "output": {
            "voice_type": "azure-standard",
            "voice": "en-US-AvaMultilingualNeural"
          }
        },
        "tool_choice": "auto",
        "tools": [
          {
            "type": "mcp",
            "server_label": "contoso_support",
            "server_url": "https://support.contoso.com/mcp",
            "project_connection_id": "<mcp-project-connection-id>",
            "allowed_tools": ["verify_customer", "get_case_status"],
            "require_approval": "never"
          },
          { "type": "system", "name": "end_conversation" },
          {
            "type": "system",
            "name": "collect_dtmf",
            "timeout_seconds": 90
          },
          {
            "type": "system",
            "name": "transfer",
            "timeout_seconds": 60,
            "targets": [
              {
                "name": "support",
                "description": "A support representative for case questions.",
                "destinations": [
                  { "type": "teams", "value": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb" }
                ],
                "routing": {
                  "strategy": "sequential",
                  "ring_timeout_seconds": 20
                }
              }
            ]
          }
        ]
      }
    }
    
  3. Save the returned agent name and version. Test the greeting in the playground before connecting the support line.

allowed_tools exposes the two business tools, and require_approval: never lets the agent invoke them during the call. Your MCP server verifies the customer before returning case information.

Connect and test the support line

Create the inbound binding, register its incoming-call URL, and test the support workflow.

Create the inbound binding

Route the Teams Phone extensibility number through your ACS connection to the saved agent.

  1. Set ACS_CONNECTION_NAME and RESOURCE_ACCOUNT_OBJECT_ID to the ACS connection name and the Teams resource-account GUID for your support number.

  2. Create the inbound binding. Replace the display phone number with your support number:

    import os
    
    from azure.ai.projects import AIProjectClient
    from azure.ai.projects.models import TeamsPhoneExtensibilityTelephonyBinding
    from azure.identity import DefaultAzureCredential
    
    with (
        DefaultAzureCredential() as credential,
        AIProjectClient(
            endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
            credential=credential,
        ) as project_client,
    ):
        binding = project_client.voice_agents.telephony.create_binding(
            agent_name="contoso-inbound-support",
            binding=TeamsPhoneExtensibilityTelephonyBinding(
                connection_name=os.environ["ACS_CONNECTION_NAME"],
                resource_account_object_id=os.environ["RESOURCE_ACCOUNT_OBJECT_ID"],
                phone_number="+14255550100",
                label="Contoso support",
            ),
        )
        print(f"Binding ID: {binding.id}")
        print(f"Incoming call URL: {binding.incoming_call_url}")
    

    Reference: AIProjectClient | DefaultAzureCredential.

  1. Create the inbound binding. Replace the resource-account GUID, ACS connection name, and display phone number with your values:

    POST {projectEndpoint}/agents/contoso-inbound-support/telephony/bindings?api-version=v1
    Authorization: Bearer <access-token>
    Content-Type: application/json
    
    {
      "provider": "teams_phone_extensibility",
      "connection_name": "contoso-acs",
      "resource_account_object_id": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
      "phone_number": "+14255550100",
      "label": "Contoso support"
    }
    

Register the incoming-call URL and test the line

After you create the binding, complete these steps for either Python or REST:

  1. Save the binding ID and incoming_call_url from the response. Use that URL to register the Event Grid subscription.
  2. Call the support number from your test phone. Follow the greeting, provide your test customer ID, and enter the code after the agent prompts you.
  3. Check both outcomes: a verified case lookup, and a request to speak with a person. Complete a separate call to test the farewell and automatic ending.

Use this sequence to verify the customer experience:

Stage What happens
Greeting The agent introduces itself and asks how it can help.
Collection The agent starts collect_dtmf, waits for listening, and asks the caller to enter the code.
Verification Private collection returns an output_ref. The agent passes the customer ID and a reference such as {{verification_code}} to verify_customer.
Case lookup The agent passes the returned verification token to get_case_status and explains its authorized summary.
Handoff or completion The agent uses transfer when a person is needed, or speaks a farewell and uses end_conversation.

Use the returned output_ref when passing the code to your business tool. See Use a private result.

Configure a transfer target

The example defines a target named support in the transfer tool. To adapt this ACS example for your support team:

  1. Set a unique name and a description that explains when to use the target.
  2. Add up to 16 destination addresses. For a Teams destination, set type to teams and value to the Teams user or resource-account object ID. For a phone-number destination, use type: phone_number and an E.164 number. Phone-number transfers also require an E.164 caller ID for the call.
  3. Choose routing.strategy: sequential tries destinations in order, and random shuffles them for each transfer.
  4. Set routing.ring_timeout_seconds to control how long each destination rings. It defaults to 20 seconds; ACS supports up to 180 seconds. Keep it within the transfer tool's timeout_seconds.

In the agent's instructions, describe when to transfer and what to say before the handoff. If the result is transfer_not_answered, offer to continue helping or try another target. For an uncertain result, have your application check the call record and provider status before requesting another transfer.

For application-controlled transfers, configure API transfer targets and manage the active call by its call ID.

Collect keypad or SMS input

Use collect_dtmf to collect keypad input on Twilio or ACS calls. For collect_sms, first configure incoming SMS for a number bound to the agent.

Set the collection tool's timeout_seconds to the time you want to allow for input. The default is 60 seconds, and the supported range is 1–300 seconds. The example uses 90 seconds.

In the agent's instructions, use secret: true and an output_variable_name for a verification code or other private value. Successful collection returns a reference that the agent can pass to your business tool. Use secret: false when the agent should read the collected value directly.

Tell the agent which keypad completion mode to use:

completion_mode Completion rule
{"type":"fixed_length","length":6} Exactly six digits arrive. length defaults to 6 and supports 1–64.
{"type":"terminator","terminator":"#"} The caller enters digits followed by #. The returned value contains the digits before the terminator. * is also supported.
{"type":"inter_digit_silence","inter_digit_timeout_seconds":2} The caller enters at least one digit, then pauses for two seconds.

The first-digit wait defaults to 15 seconds, and the pause between digits defaults to 5 seconds. To adjust them, specify first_digit_timeout_seconds in the collection arguments and inter_digit_timeout_seconds inside completion_mode. Collected strings preserve leading zeros.

For private SMS collection, instruct the agent to use the caller's SMS number as from_phone_number. For example, the tool arguments can be:

{
  "secret": true,
  "from_phone_number": "+14255550123",
  "output_variable_name": "verification_code"
}

Use the sender's E.164 number in from_phone_number. The tool collects one numeric message from that number, and your business service verifies the code.

Instruct the agent to ask for input after collection reports listening, then wait for the result. Offer a retry or support transfer if collection expires or fails.

Configure incoming SMS

To collect a numeric message during a call, route incoming SMS from your Twilio number to the agent:

  1. Bind an SMS-capable Twilio number to the agent.

  2. In the Twilio number's Messaging settings, set A message comes in to Webhook and select HTTP POST. Use this URL:

    {projectEndpoint}/agents/{agentName}/endpoint/protocols/voice/telephony/twilio/callbacks/collection-sms?api-version=v1&phone={encodedAgentPhoneNumber}
    

    Use your Foundry project endpoint and agent name. Set phone to the URL-encoded bound number; for example, +14255550100 becomes %2B14255550100.

  3. If the number belongs to a Twilio Messaging Service, configure that service to use the phone number's incoming-message webhook. Save the settings.

During the call, the agent starts collect_sms with the sender's number in from_phone_number. After the tool reports listening, ask the caller to send their numeric message to the bound agent number.

Use a private result

A successful private collection returns a name in output_ref. Wrap that returned name in double braces when passing it to an MCP tool, send_sms, or send_dtmf. For example, if output_ref is verification_code, pass:

{
  "code": "{{verification_code}}"
}

The service supplies the private value to the receiving tool while the agent sees the reference. Have your business tool return a verification result and keep the private value out of its response and logs.

Send tones or an SMS

Enable send_dtmf to navigate a telephone menu or send_sms to send a text during a call. Describe when to use the tool and what to send in the agent's instructions.

Tool Arguments to describe in the instructions
send_dtmf digits, such as 1# for a telephone menu selection. Use 0–9, *, and #.
send_sms to_phone_number, the recipient's E.164 number, and message, the text to send.

For send_dtmf, send 1–18 tones per invocation on ACS or 1–64 on Twilio and SIP. To send a private value, use its returned reference, such as {{verification_code}}#.

For a Twilio call, bind an SMS-capable number to the agent. The send_sms tool sends from that bound number. Set timeout_seconds in either tool's configuration to control how long the action can take.

Observe execution and test

Use separate test calls for the following scenarios. Review call activity in call records and dialogue in stored conversations.

  • Complete a successful verification and case lookup. Confirm that the caller hears the correct summary and your MCP service receives the verification input.
  • Test an incorrect code and a collection timeout. Confirm that the agent offers a retry or a support transfer.
  • Test a transfer to the support recipient and a call where that recipient is unavailable. Check that the agent responds appropriately to each outcome.
  • Complete the caller's request and confirm that the farewell finishes before the call ends.

If your application reads voice protocol events, use input_collection.status to inspect collection progress and phone-action feedback. The event's call_id identifies the tool invocation. Use the telephony call ID for Calls API requests. A system_tool item's completed status describes argument generation. Use the execution feedback and call record to check the action's result.

Outcome Meaning and next step
provider_accepted The provider accepted the request. Confirm SMS delivery or the telephone menu's response through the provider or conversation flow.
transfer_not_answered The transfer target didn't answer, and the original conversation remains available. Offer to continue helping or retry at the caller's request.
provider_not_executed The service stopped before sending or dialing. Review the cause before starting a new request.
provider_failed_or_unsupported The action failed validation or execution. Check the tool arguments and provider configuration.
provider_result_uncertain Check the call record and provider status to establish the outcome before requesting the action again.

After testing, remove temporary bindings that you no longer need.

Troubleshoot tool availability

Use the symptom to choose what to check:

Symptom What to check
The agent skips a tool Confirm that the saved tools array includes the tool, tool_choice is auto, and the instructions describe when to use the tool. Start a new session with the updated version.
Keypad collection times out Check that the agent waits for listening before prompting and that the caller follows the configured completion mode.
A transfer fails Check the target's name and destination, the configured caller ID, and whether the recipient can receive the call.
SMS sending fails Check the sender number's SMS capability, its binding to the agent, and the provider result.

For call-query examples and an investigation workflow, see Troubleshoot telephony.

Continue with these guides to configure the agent or create outbound calling workflows: