Visualize Azure AI Search Logs and Metrics with Power BI
Azure AI Search can send operation logs and service metrics to an Azure Storage account, which you can then visualize in Power BI. This article explains the steps and how to use a Power BI Template App to visualize the data. The template can help you gain detailed insights about your search service, including information about queries, indexing, operations, and service metrics.
You can find the Power BI Template App Azure AI Search: Analyze Logs and Metrics in the Power BI Apps marketplace.
Set up the app
Enable metric and resource logging for your search service:
Create or identify an existing Azure Storage account where you can archive the logs.
Navigate to your Azure AI Search service in the Azure portal.
Under the Monitoring section on the left column, select Diagnostic settings.
Select + Add diagnostic setting.
Check Archive to a storage account, provide your Storage account information, and check OperationLogs and AllMetrics.
After logging has been enabled, use your search service to start generating logs and metrics. It takes up to an hour before the containers will appear in Blob storage with these logs. You will see a insights-logs-operationlogs container for search traffic logs and a insights-metrics-pt1m container for metrics.
Find the Azure AI Search Power BI App in the Power BI Apps marketplace and install it into a new workspace or an existing workspace. The app is called Azure AI Search: Analyze Logs and Metrics.
After installing the app, select the app from your list of apps in Power BI.
Select Connect to connect your data
Input the name of the storage account that contains your logs and metrics. By default the app will look at the last 10 days of data but this value can be changed with the Days parameter.
Select Key as the authentication method and provide your storage account key. Select Private as the privacy level. Click Sign In and to begin the loading process.
Wait for the data to refresh. This may take some time depending on how much data you have. You can see if the data is still being refreshed based on the below indicator.
Once the data refresh has completed, select Azure AI Search Report to view the report.
Make sure to refresh the page after opening the report so that it opens with your data.
Modify app parameters
If you would like to visualize data from a different storage account or change the number of days of data to query, follow the below steps to change the Days and StorageAccount parameters.
Navigate to your Power BI apps, find your Azure AI Search app and select the Edit app button to view the workspace.
Select Settings from the Dataset options.
While in the Datasets tab, change the parameter values and select Apply. If there is an issue with the connection, update the data source credentials on the same page.
Navigate back to the workspace and select Refresh now from the Dataset options.
Open the report to view the updated data. You might also need to refresh the report to view the latest data.
Troubleshooting report issues
If you find that you cannot see your data follow these troubleshooting steps:
Open the report and refresh the page to make sure you're viewing the latest data. There's an option in the report to refresh the data. Select this to get the latest data.
Ensure the storage account name and access key you provided are correct. The storage account name should correspond to the account configured with your search service logs.
Confirm that your storage account contains the containers insights-logs-operationlogs and insights-metrics-pt1m and each container has data. The logs and metrics will be within a couple layers of folders.
Check to see if the dataset is still refreshing. The refresh status indicator is shown in step 8 above. If it is still refreshing, wait until the refresh is complete to open and refresh the report.