Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
This article explains how to capture telemetry data from your Copilot Studio agent for use in Azure Application Insights.
In addition to the native analytics features within Copilot Studio, you can send telemetry data to Application Insights. Telemetry offers insights into your agent by tracking:
- Logged messages and events sent to and from your agent
- Topics to trigger during user conversations
- Custom telemetry events that you can send from your topics
Important
Application Insights is a feature of Azure Monitor, an extensible Application Performance Management (APM) tool that you can use to monitor your live applications. It requires a subscription to Microsoft Azure.
Connect your Copilot Studio agent to Application Insights
To connect your agent to Application Insights, first add your instrumentation key to your agent's configuration.
Go to the Settings page for your agent, and select Advanced.
Within the Application Insights section, enter the Connection string. To learn how to locate your connection string, see the Azure Monitor documentation.
Optionally, you can choose to enable one of the following settings.
Log activities: If enabled, the system logs details of incoming and outgoing messages and events.
Log sensitive Activity properties: If enabled, the logs include the values of certain properties that could be considered sensitive on incoming and outgoing messages and events. The properties that are considered potentially sensitive are
userid,name,text, andspeak(text and speak properties only apply to messages).
Analyze bot telemetry with Application Insights
After you connect your bot to Application Insights, it logs telemetry data when users interact with the bot, including during testing within Copilot Studio. To see the logged telemetry data, go to the Logs section of your Application Insights resource in Azure.
From here, you can use Kusto queries to query and analyze your data. See example queries.
Example queries
A query can be as simple as specifying a single table, such as customEvents, which shows all custom telemetry events logged from Copilot Studio. But you can also use Kusto queries to narrow down your results further, including:
- Adding a time interval
- Extending your results by using custom dimensions. Custom dimensions are custom properties that you log along with the prebuilt fields, such as timestamp or the event name
- Adding a where clause to limit the data returned based on a condition
- Using more built-in Kusto functions to determine what and how information is shown
The following example query results in a line chart that shows how many distinct users communicated with your bot per day for the past 14 days.
let queryStartDate = ago(14d);
let queryEndDate = now();
let groupByInterval = 1d;
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| summarize uc=dcount(user_Id) by bin(timestamp, groupByInterval)
| render timechart
Important
The data within some fields varies and is more or less applicable, depending on the channel that is being used. For example, you only get a correct count of unique users in the query if the users are authenticated and their user IDs are consistent across conversations. In anonymous scenarios where a random user ID is generated in each conversation, the user ID field is less useful.
Exclude telemetry from test conversations in your queries
Your bot logs telemetry for all conversations, including those that happen within Copilot Studio during testing. If you want to exclude telemetry gathered during testing, extend your query by using the designMode custom dimension that all events capture, and use a where clause in your query.
The following example shows all custom events, excluding those captured through the test canvas.
customEvents
| extend isDesignMode = customDimensions['designMode']
| where isDesignMode == "False"
Custom Dimensions
Much of the specific activity data received from Copilot Studio is stored in the customDimensions field. You can see a custom dimension field being used in a query to exclude telemetry from test conversations.
| Field | Description | Sample Values |
|---|---|---|
| type | Type of activity | message, conversationUpdate, event, invoke |
| channelId | Channel identifier | emulator, directline, msteams, webchat |
| fromId | From Identifier | <id> |
| fromName | Username from client | John Bonham, Keith Moon, Steve Smith, Steve Gadd |
| locale | Client origin locale | en-us, zh-cn, en-GB, de-de, zh-CN |
| recipientId | Recipient identifier | <id> |
| recipientName | Recipient name | John Bonham, Keith Moon, Steve Smith, Steve Gadd |
| text | Text in message | find a coffee shop |
| designMode | Conversation happened within the test canvas | True / False |
Viewing monitoring results (Preview)
Important
This article contains Microsoft Copilot Studio preview documentation and is subject to change.
Preview features aren't meant for production use and may have restricted functionality. These features are available before an official release so that you can get early access and provide feedback.
If you're building a production-ready agent, see Microsoft Copilot Studio Overview.
The Copilot Studio dashboard view uses signals from Azure Monitor Application Insights. It queries Application Insights through Azure Workbooks and creates visualizations.
These views bring key metrics, such as total conversations, latency, exceptions, tool usage, and topic analytics, into a single view. The single view provides teams with transparency so that they can track operational health and quality, understand trends, and assess continuously to improve their application.
To access and use the built-in monitoring view in Application Insights, follow these steps:
- Go to your Application Insights resource.
- Select the Monitoring tab from the left navigation pane.
- Under the Monitoring tab, select Workbooks. Open Copilot Studio Dashboard from the workbooks gallery.
Customize and share your dashboard
Application Insights is a powerful tool for application performance monitoring (APM) that provides insights into the health and performance of your applications.
The dashboard opens as an editable workbook where you can customize the workbook and save it according to your needs.
Select Edit in the command bar.
Modify elements as needed for your use case. Select the three dots (…) for an element to edit, add, move or resize, clone, or remove it. For example, you can add a tile that uses KQL to track a custom attribute you're collecting that the built-in view doesn't show.
Select Save to save your latest changes and create different views as needed.
Select the Share icon in the command bar to share with your team.
Note
When you share this workbook with your team members, they must have at least the Reader role for the connected Application Insights resource to view the displayed information.
