Quickstart: Create a browser-enabled prompt agent

Create a Foundry prompt agent with the Browser automation tool, run a browser task, and verify the result. Foundry runs the agent; you don't deploy a custom agent runtime.

Prerequisites

  • An Azure subscription. Create one for free.

  • Foundry User role on the Foundry project for day-to-day agent development and use.

    Important

    The Foundry RBAC roles were recently renamed. Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously named Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. You might still see the previous names in some places while the rename rolls out. The role IDs and core permissions are unchanged by the rename.

  • Foundry Project Manager role on the Foundry project if you create the project connection.

  • Contributor role on the target resource group only while you create the Playwright workspace. This role is required for resource provisioning. Activate it just in time through Microsoft Entra Privileged Identity Management (PIM), and deactivate it after provisioning. Day-to-day agent developers and runtime users don't need this role.

  • A Foundry project with a configured endpoint.

  • An AI model deployed in your project (for example, gpt-5.4). Confirm that both the model and project region support Browser Automation in Tool support by region and model.

  • A Playwright workspace resource.

  • A project connection set up for your Playwright workspace.

SDK requirements

For Python examples, install the required packages:

pip install "azure-ai-projects>=2.8.0" python-dotenv

The .NET SDK is currently in preview. For more information, see the quickstart.

Configuration

Get your project endpoint: Open your project in the Foundry portal, and copy the endpoint from the project overview page. The format is https://{account-name}.services.ai.azure.com/api/projects/{project-name}. Connection ID format: Use /subscriptions/{{subscriptionID}}/resourceGroups/{{resourceGroupName}}/providers/Microsoft.CognitiveServices/accounts/{{foundryAccountName}}/projects/{{foundryProjectName}}/connections/{{foundryConnectionName}}. You can find this value on the tool's details page after you connect the Browser Automation tool.

Step 1: Create a Playwright workspace

  1. In the Azure portal, create a Playwright Workspace resource.
  2. After the workspace is created, go to Settings > Access Management.

If you selected API token as the authentication type:

  1. Confirm the Playwright Service Access Token authentication method is enabled.
  2. Select Generate Token, enter a name (for example, foundry-connection), and choose an expiry period.
  3. Copy the token immediately. You can't view it again after closing the page.
  4. Store the token only in the Foundry project connection. Don't put it in source code, prompts, or application logs. Rotate it before it expires, and revoke it immediately if it is exposed.

On the workspace Overview page, copy the Browser endpoint (it starts with wss://). Next, grant the Foundry project managed identity access to the Playwright workspace.

Step 2: Grant Playwright workspace access

Browser Automation Tool uses the Foundry project managed identity to access the Playwright workspace. This identity is different from a hosted agent's dedicated agent identity:

  • Project managed identity: Project-wide identity used by Foundry platform services and Browser Automation connections.
  • Agent identity: Per-agent identity created when you deploy a hosted agent. Use it for the agent's runtime access to downstream resources.

Grant the project managed identity the minimum permissions required to use the Playwright workspace:

  1. In the Azure portal, open the Playwright workspace.
  2. Select Access control (IAM), and then select Add > Add role assignment.
  3. Select a custom role that contains only the Playwright permissions required by the Foundry project identity. If a custom role isn't available, assign Playwright Workspace Contributor at the Playwright workspace resource scope.
  4. On the Members tab, select the Foundry project managed identity.
  5. Select Review + assign.

Step 3: Configure Browser Automation in Foundry

Configure a Browser Automation connection in your Foundry project before you run the agent samples. The connection associates Browser Automation with your Playwright workspace and provides the connection information required by the SDK samples.

  1. Go to the Foundry portal and select your project.
  2. Create a Browser Automation connection to the Playwright workspace that you configured earlier.
  3. Provide the required connection details:
    • Connection name: Enter a unique name for the connection.
    • Playwright Workspace: Select your Playwright workspace resource.
    • Auth Type: Select the authentication type for the connection.
  4. Save the connection.

The agent samples use this connection to configure the Browser Automation tool. Depending on the SDK, the sample either retrieves the connection by name or uses the project connection ID.

Step 4: Use Browser Automation Tool with agents example

import os
import json
from dotenv import load_dotenv
from util import create_version_with_endpoint
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import (
    PromptAgentDefinition,
    BrowserAutomationTool,
    BrowserAutomationToolParameters,
    BrowserAutomationToolConnectionParameters,
)

load_dotenv()

endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"]
agent_name = os.environ.get("FOUNDRY_AGENT_NAME") or "MyAgent"


tool = BrowserAutomationTool(
    browser_automation=BrowserAutomationToolParameters(
        connection=BrowserAutomationToolConnectionParameters(
            project_connection_id=os.environ["BROWSER_AUTOMATION_PROJECT_CONNECTION_ID"],
        )
    )
)

with (
    DefaultAzureCredential() as credential,
    AIProjectClient(endpoint=endpoint, credential=credential) as project_client,
    create_version_with_endpoint(
        project_client=project_client,
        agent_name=agent_name,
        definition=PromptAgentDefinition(
            model=os.environ["FOUNDRY_MODEL_NAME"],
            instructions="""You are an Agent helping with browser automation tasks.
        You can answer questions, provide information, and assist with various tasks
        related to web browsing using the Browser Automation tool available to you.""",
            tools=[tool],
        ),
    ),
    project_client.get_openai_client(agent_name=agent_name) as openai_client,
):
    stream_response = openai_client.responses.create(
        stream=True,
        input="""
        Use the Browser Automation tool to visit https://example.com.
        Read the page and report its title and the visible description paragraph.
        Base your answer on the browser results, not on prior knowledge.""",
    )

    for event in stream_response:
        if event.type == "response.created":
            print(f"Response created with ID: {event.response.id}")
        elif event.type == "response.output_item.done":
            item = event.item
            # Each browser action (navigate, click, type, ...) is surfaced as a "remote_function_call" item.
            if item.type == "remote_function_call":
                arguments = json.loads(getattr(item, "arguments", None) or "{}")
                arguments.pop("browserSessionId", None)
                print(f"Browser action: {item.name} (call ID: {item.call_id}), arguments: {arguments}")
        elif event.type == "response.output_text.done":
            # Includes the live view URL message as well as the final answer.
            print(f"\nAgent message: {event.text}")
        elif event.type == "response.completed":
            print("\nResponse completed!")

What this code does

This example creates an agent version with the browser automation tool enabled and sends a request that requires the agent to interact with a webpage. The example streams the response and reports browser actions as they occur.

At the start of the run, the agent provides a Live View URL that you can use to observe the browser session. Treat this URL as sensitive and don't share it.

Required inputs

Set the following environment variables before running the sample:

  • FOUNDRY_PROJECT_ENDPOINT: Your Microsoft Foundry project endpoint.
  • FOUNDRY_MODEL_NAME: The deployment name of the AI model in your Foundry project.
  • BROWSER_AUTOMATION_PROJECT_CONNECTION_ID: The Browser Automation project connection ID.
  • FOUNDRY_AGENT_NAME: Optional. The name of the agent. If you don't specify a value, the sample uses MyAgent.

See Configuration for details.

Expected output

When you run the sample, you see streaming updates as the agent completes the browser task. The output can include:

  • The response ID.
  • A Live View URL for observing the browser session.
  • Browser actions performed by the agent.
  • The agent's final response.
  • A message indicating that the response completed.

The exact output varies based on the website content and model behavior.

This example uses synchronous methods of the Azure AI Projects client library. For an example that uses asynchronous methods, see the Sample for use of BrowserAutomationTool and Agents sample in the Azure SDK for .NET repository on GitHub.

  1. Create an AIProjectClient and read the required environment variables. Browser automation operations can take longer than other requests, so configure a network timeout of at least five minutes.
var projectEndpoint = System.Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT");
var modelDeploymentName = System.Environment.GetEnvironmentVariable("FOUNDRY_MODEL_NAME");
var playwrightConnectionName = System.Environment.GetEnvironmentVariable("BROWSER_AUTOMATION_NAME");
AIProjectClientOptions options = new()
{
    NetworkTimeout = TimeSpan.FromMinutes(5)
};
AIProjectClient projectClient = new(endpoint: new Uri(projectEndpoint), tokenProvider: new DefaultAzureCredential(), options: options);
  1. Look up the Browser Automation connection by name, create an agent with a BrowserAutomationTool, and pass the returned connection ID to BrowserAutomationToolConnectionOptions.

Synchronous sample:

AIProjectConnection playwrightConnection = projectClient.Connections.GetConnection(playwrightConnectionName);
BrowserAutomationTool playwrightTool = new(
    new BrowserAutomationToolOptions(
        new BrowserAutomationToolConnectionOptions(playwrightConnection.Id)
    ));

DeclarativeAgentDefinition agentDefinition = new(model: modelDeploymentName)
{
    Instructions = "You are an Agent helping with browser automation tasks.\n" +
    "You can answer questions, provide information, and assist with various tasks\n" +
    "related to web browsing using the Browser Automation tool available to you.",
    Tools = { playwrightTool }
};
ProjectsAgentVersion agentVersion = projectClient.AgentAdministrationClient.CreateAgentVersion(
    agentName: "myAgent",
    options: new(agentDefinition));

Asynchronous sample:

AIProjectConnection playwrightConnection = await projectClient.Connections.GetConnectionAsync(playwrightConnectionName);
BrowserAutomationTool playwrightTool = new(
    new BrowserAutomationToolOptions(
        new BrowserAutomationToolConnectionOptions(playwrightConnection.Id)
    ));

DeclarativeAgentDefinition agentDefinition = new(model: modelDeploymentName)
{
    Instructions = "You are an Agent helping with browser automation tasks.\n" +
    "You can answer questions, provide information, and assist with various tasks\n" +
    "related to web browsing using the Browser Automation tool available to you.",
    Tools = { playwrightTool }
};
ProjectsAgentVersion agentVersion = await projectClient.AgentAdministrationClient.CreateAgentVersionAsync(
    agentName: "myAgent",
    options: new(agentDefinition));
  1. Create a ParseResponse helper method to process streaming response updates from the agent. Handle failed or incomplete responses using the SDK's typed updates.

Warning: Streaming text and complete messages can include a live-view URL that grants access to the browser session. Treat that URL as sensitive: use it only in a trusted local terminal and redact it before sharing output or saving logs.

private static void ParseResponse(StreamingResponseUpdate streamResponse)
{
    if (streamResponse is StreamingResponseCreatedUpdate createUpdate)
    {
        Console.WriteLine($"Stream response created with ID: {createUpdate.Response.Id}");
    }
    else if (streamResponse is StreamingResponseOutputTextDeltaUpdate textDelta)
    {
        Console.WriteLine($"Delta: {textDelta.Delta}");
    }
    else if (streamResponse is StreamingResponseOutputTextDoneUpdate textDoneUpdate)
    {
        Console.WriteLine($"Response done with full message: {textDoneUpdate.Text}");
    }
    else if (streamResponse is StreamingResponseErrorUpdate errorUpdate)
    {
        throw new InvalidOperationException($"The stream has failed: {errorUpdate.Message}, error code: {errorUpdate.Code}");
    }
    else if (streamResponse is StreamingResponseFailedUpdate failedUpdate)
    {
        throw new InvalidOperationException($"The response has failed: {failedUpdate.Response.Error?.Message}, error code: {failedUpdate.Response.Error?.Code}");
    }
    else if (streamResponse is StreamingResponseIncompleteUpdate incompleteUpdate)
    {
        throw new InvalidOperationException($"The response cannot be completed. Check the request for safety or guardrail requirements and review response {incompleteUpdate.Response.Id} for details, including output limits.");
    }
}
  1. Create a streaming response. Require the agent to use a tool by setting ToolChoice = ResponseToolChoice.CreateRequiredChoice() on CreateResponseOptions.

Synchronous sample:

ProjectResponsesClient responseClient = projectClient.ProjectOpenAIClient.GetProjectResponsesClientForAgent(
    new AgentReference(agentVersion.Name, agentVersion.Version));
CreateResponseOptions responseOptions = new()
{
    ToolChoice = ResponseToolChoice.CreateRequiredChoice(),
    StreamingEnabled = true,
    InputItems =
    {
        ResponseItem.CreateUserMessageItem("Your goal is to report the percent of Microsoft year-to-date stock price change.\n" +
            "To do that, go to the website finance.yahoo.com.\n" +
            "At the top of the page, you will find a search bar.\n" +
            "Enter the value 'MSFT', to get information about the Microsoft stock price.\n" +
            "At the top of the resulting page you will see a default chart of Microsoft stock price.\n" +
            "Click on 'YTD' at the top of that chart, and report the percent value that shows up just below it.")
    }
};
foreach (StreamingResponseUpdate update in responseClient.CreateResponseStreaming(responseOptions))
{
    ParseResponse(update);
}

Asynchronous sample:

ProjectResponsesClient responseClient = projectClient.ProjectOpenAIClient.GetProjectResponsesClientForAgent(
    new AgentReference(agentVersion.Name, agentVersion.Version));
CreateResponseOptions responseOptions = new()
{
    ToolChoice = ResponseToolChoice.CreateRequiredChoice(),
    StreamingEnabled = true,
    InputItems =
    {
        ResponseItem.CreateUserMessageItem("Your goal is to report the percent of Microsoft year-to-date stock price change.\n" +
            "To do that, go to the website finance.yahoo.com.\n" +
            "At the top of the page, you will find a search bar.\n" +
            "Enter the value 'MSFT', to get information about the Microsoft stock price.\n" +
            "At the top of the resulting page you will see a default chart of Microsoft stock price.\n" +
            "Click on 'YTD' at the top of that chart, and report the percent value that shows up just below it.")
    }
};
await foreach (StreamingResponseUpdate update in responseClient.CreateResponseStreamingAsync(responseOptions))
{
    ParseResponse(update);
}
  1. After the sample completes, delete the agent version that you created. Agents can be reused across responses. Delete an agent version only when it is no longer needed.

Synchronous sample:

projectClient.AgentAdministrationClient.DeleteAgentVersion(agentName: agentVersion.Name, agentVersion: agentVersion.Version);

Asynchronous sample:

await projectClient.AgentAdministrationClient.DeleteAgentVersionAsync(agentName: agentVersion.Name, agentVersion: agentVersion.Version);

What this code does

This example creates a Foundry agent with the Browser Automation tool, retrieves the Browser Automation connection by name, and sends the agent a browser-based task. The example streams response updates while the agent uses the configured browser connection to interact with the target website.

Required inputs

Set the following environment variables before running the sample:

  • FOUNDRY_PROJECT_ENDPOINT: Your Microsoft Foundry project endpoint.
  • FOUNDRY_MODEL_NAME: The deployment name of the AI model in your Foundry project.
  • BROWSER_AUTOMATION_NAME: The name of the Browser Automation connection in your Foundry project.

The sample retrieves the Browser Automation connection by name and uses the returned connection ID to configure BrowserAutomationTool.

See Configuration for details.

Expected output

When you run the sample, the console displays streaming response updates, including the response ID, text deltas, and the completed message. The response can include a Live View URL for the session. Output varies based on website content and model behavior.

const { DefaultAzureCredential } = require("@azure/identity");
const { AIProjectClient } = require("@azure/ai-projects");
require("dotenv/config");

const projectEndpoint = process.env["FOUNDRY_PROJECT_ENDPOINT"] || "<project endpoint>";
const deploymentName = process.env["FOUNDRY_MODEL_NAME"] || "<model deployment name>";
const agentName = process.env["FOUNDRY_AGENT_NAME"] || "MyAgent";
const browserAutomationProjectConnectionId =
  process.env["BROWSER_AUTOMATION_PROJECT_CONNECTION_ID"] ||
  "<browser automation project connection id>";

// Foundry remote function items are not part of OpenAI's ResponseOutputItem union.
const handleBrowserCall = (item) => {
  if (typeof item !== "object" || item === null || !("type" in item)) {
    return;
  }
  if (item.type === "remote_function_call") {
    if (
      !("name" in item) ||
      typeof item.name !== "string" ||
      !("call_id" in item) ||
      typeof item.call_id !== "string" ||
      !("arguments" in item) ||
      typeof item.arguments !== "string"
    ) {
      throw new Error("Invalid Browser Automation remote function call.");
    }
    const argumentsObj = JSON.parse(item.arguments);
    const argumentsStr = JSON.stringify(argumentsObj, (key, value) =>
      key === "browserSessionId" ? undefined : value,
    );
    console.log(
      `Browser action: ${item.name} (call ID: ${item.call_id}), arguments: ${argumentsStr}`,
    );
  } else if (item.type === "remote_function_call_output") {
    if (!("call_id" in item) || typeof item.call_id !== "string") {
      throw new Error("Invalid Browser Automation remote function call output.");
    }
    console.log(`Browser action output received for call ID: ${item.call_id}`);
  }
};

async function main() {
  const project = new AIProjectClient(projectEndpoint, new DefaultAzureCredential());
  const openAIClient = project.getOpenAIClient();

  console.log("Creating agent with Browser Automation tool...");

  const agent = await project.agents.createVersion(agentName, {
    kind: "prompt",
    model: deploymentName,
    instructions: `You are an Agent helping with browser automation tasks. 
            You can answer questions, provide information, and assist with various tasks 
            related to web browsing using the Browser Automation tool available to you.`,
    // Define Browser Automation tool
    tools: [
      {
        type: "browser_automation",
        browser_automation: {
          connection: {
            project_connection_id: browserAutomationProjectConnectionId,
          },
        },
      },
    ],
  });
  console.log(`Agent created (id: ${agent.id}, name: ${agent.name}, version: ${agent.version})`);

  try {
    console.log("\nSending browser automation request with streaming...");
    const streamResponse = await openAIClient.responses.create(
      {
        input:
          "Use the Browser Automation tool to visit https://example.com. Read the page and report its title and the visible description paragraph. Base your answer on the browser results, not on prior knowledge.",
        stream: true,
      },
      {
        body: {
          agent_reference: { name: agent.name, version: agent.version, type: "agent_reference" },
        },
      },
    );

    console.log(
      "The browser live-view URL is sensitive. Do not share it or commit it to source control.",
    );
    let responseCompleted = false;
    for await (const event of streamResponse) {
      if (event.type === "response.created") {
        console.log(`Response created with ID: ${event.response.id}`);
      } else if (event.type === "response.output_text.done") {
        console.log(`\nAgent message: ${event.text}`);
      } else if (event.type === "response.output_item.done") {
        handleBrowserCall(event.item);
      } else if (event.type === "response.failed") {
        throw new Error(event.response.error?.message ?? "Browser Automation response failed.");
      } else if (event.type === "response.incomplete") {
        throw new Error(
          `Browser Automation response incomplete: ${event.response.incomplete_details?.reason}`,
        );
      } else if (event.type === "error") {
        throw new Error(event.message);
      } else if (event.type === "response.completed") {
        responseCompleted = true;
        console.log("\nResponse completed!");
      }
    }
    if (!responseCompleted) {
      throw new Error("Browser Automation stream ended without a completed response.");
    }
  } finally {
    console.log("\nCleaning up resources...");
    await project.agents.deleteVersion(agent.name, agent.version);
    console.log("Agent deleted");
  }

  console.log("\nBrowser Automation sample completed!");
}

main().catch((err) => {
  console.error("The sample encountered an error:", err);
  process.exitCode = 1;
});

module.exports = { main };

What this code does

This example creates an agent version with the browser automation tool enabled and sends a request that requires the agent to interact with a webpage. The example streams the response and reports browser actions as they occur.

At the start of the run, the agent provides a Live View URL that you can use to observe the browser session. Treat this URL as sensitive and don't share it.

Required inputs

Set the following environment variables before running the sample:

  • FOUNDRY_PROJECT_ENDPOINT: Your Microsoft Foundry project endpoint.
  • FOUNDRY_MODEL_NAME: The deployment name of the AI model in your Foundry project.
  • BROWSER_AUTOMATION_PROJECT_CONNECTION_ID: The Browser Automation project connection ID.
  • FOUNDRY_AGENT_NAME: Optional. The name of the agent. If you don't specify a value, the sample uses MyAgent.

See Configuration for details.

Expected output

When you run the sample, you see streaming updates as the agent completes the browser task. The output can include:

  • The response ID.
  • A Live View URL for observing the browser session.
  • Browser actions performed by the agent.
  • The agent's final response.
  • A message indicating that the response completed.

The exact output varies based on the website content and model behavior.

Step 5: Clean up

Delete test agent versions when you finish using them. If a sample automatically deletes its agent version, you won't find it later in the portal.

Close browser sessions when you see explicit cleanup instructions. Remove sample-only toolbox versions and connections if you no longer need them. Revoke test access tokens. Delete an unused sample workspace only after checking that other agents don't depend on it.