Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
This article discusses how you can capture telemetry data from your Copilot Studio agent for use in Azure Application Insights.
In addition to the native analytics features within Copilot Studio, you can send telemetry data to Application Insights. Telemetry offers insights into your agent by tracking:
- Logged messages and events sent to and from your agent
- Topics to be triggered during user conversations
- Custom telemetry events that can be sent from your topics
Important
Application Insights is a feature of Azure Monitor, an extensible Application Performance Management (APM) tool that allows you to monitor your live applications. It requires a subscription to Microsoft Azure.
Connect your Copilot Studio agent to Application Insights
To connect your agent to Application Insights, you first need to add your instrumentation key to your agent's configuration.
Go to the Settings page for your agent, and select Advanced.
Within the Application Insights section, populate the Connection string setting. See the Azure Monitor documentation to find out how to locate your connection string.
Optionally, you can choose to enable one of the following settings.
Log activities: If enabled, details of incoming/outgoing messages and events are logged.
Log sensitive Activity properties: If enabled, the values of certain properties that could be considered sensitive on incoming/outgoing messages and events are included in logs. The properties that are considered potentially sensitive are userid, name, text and speak (text and speak properties only apply to messages).
Analyze bot telemetry with Application Insights
After you connect your bot to Application Insights, telemetry data is logged when users interact with the bot, including during testing within Copilot Studio. To see the logged telemetry data, navigate to the Logs section of your Application Insights resource in Azure.
From here, you can use Kusto queries to query and analyze your data. See example queries.
Example queries
A query can be as simple as specifying a single table, such as customEvents, which shows all custom telemetry events logged from Copilot Studio. But you can also use Kusto queries to narrow down your results further, including;
- Adding a time interval
- Extending your results using custom dimensions. Custom dimensions are custom properties that are logged, along with the prebuilt fields, such as timestamp or the event name
- A where clause to limit the data returned based on a condition
- Using more built-in Kusto functions to determine what and how information is shown
The example query below results in a line chart that shows how many distinct users communicated with your bot per day for the past 14 days.
let queryStartDate = ago(14d);
let queryEndDate = now();
let groupByInterval = 1d;
customEvents
| where timestamp > queryStartDate
| where timestamp < queryEndDate
| summarize uc=dcount(user_Id) by bin(timestamp, groupByInterval)
| render timechart
Important
The data within some fields varies and is more or less applicable, depending on the channel that is being used. For example, you only get a correct count of unique users in the query if the users are authenticated and their user ids are consistent across conversations. In anonymous scenarios where a random user id is generated in each conversation, the user id field is less useful.
Excluding telemetry from test conversations from your queries
Your bot logs telemetry for all conversations, including those that happen within Copilot Studio during testing. If you want to exclude telemetry gathered during testing, you can extend your query with the designMode custom dimension that is captured on all events, and use a where clause in your query.
The following example shows all custom events, excluding those captured through the test canvas.
customEvents
| extend isDesignMode = customDimensions['designMode']
| where isDesignMode == "False"
Custom Dimensions
Much of the specific activity data received from Copilot Studio is stored in the customDimensions field. You can see a custom dimension field being used in a query to exclude telemetry from test conversations.
| Field | Description | Sample Values |
|---|---|---|
| type | Type of activity | message, conversationUpdate, event, invoke |
| channelId | Channel identifier | emulator, directline, msteams, webchat |
| fromId | From Identifier | <id> |
| fromName | Username from client | John Bonham, Keith Moon, Steve Smith, Steve Gadd |
| locale | Client origin locale | en-us, zh-cn, en-GB, de-de, zh-CN |
| recipientId | Recipient identifier | <id> |
| recipientName | Recipient name | John Bonham, Keith Moon, Steve Smith, Steve Gadd |
| text | Text in message | find a coffee shop |
| designMode | Conversation happened within the test canvas | True / False |
Viewing monitoring results (Preview)
Important
This article contains Microsoft Copilot Studio preview documentation and is subject to change.
Preview features aren't meant for production use and may have restricted functionality. These features are available before an official release so that you can get early access and provide feedback.
If you're building a production-ready agent, see Microsoft Copilot Studio Overview.
The Copilot Studio dashboard view uses signals from Azure Monitor Application Insights, querying it through Azure Workbooks and creating visualizations.
These views are designed to bring key metrics, such as total conversations, latency, exceptions, tool usage, and topic analytics into a single view. The single view provides teams with transparency so that they can track operational health and quality, understand trends, and assess continuously to improve their application.
Follow these steps to access and utilize the built-in monitoring view in Application Insights:
- Navigate to your Application Insights.
- Select the Monitoring tab from the left navigation pane.
- Under the Monitoring tab, select Workbooks and open Copilot Studio Dashboard from the workbooks gallery.
Customize and share your dashboard
Application Insights is a powerful tool for application performance monitoring (APM) that provides insights into the health and performance of your applications.
The dashboard opens as an editable workbook where you can customize the workbook and save according to your needs.
- Select Edit in the command bar.
- Modify elements as needed for your use case. Select ... on an element to edit, add, move or resize, clone, or remove. For example, you can add a tile using KQL to track a custom attribute you're collecting that is not shown in our built-in view.
- Save your latest changes and create different views as needed by selecting Save.
- Share with your team by selecting the Share icon in the command bar.
Note
When you share this workbook with your team members, they must have at least Reader role for the connected Application Insights resource in order to view the displayed information.
