Sources of monitoring data for Azure Monitor
Azure Monitor is based on a common monitoring data platform that includes
This platform allows data from multiple resources to be analyzed together using a common set of tools in Azure Monitor. Monitoring data may also be sent to other locations to support certain scenarios, and some resources may write to other locations before they can be collected into Logs or Metrics.
This article describes common sources of monitoring data collected by Azure Monitor in addition to the monitoring data created by Azure resources. Links are provided to detailed information on configuration required to collect this data to different locations.
Some of these data sources use the new data ingestion pipeline in Azure Monitor. This article will be updated as other data sources transition to this new data collection method.
Note
Access to data in the Log Analytics Workspaces is governed as outline here.
Application tiers
Sources of monitoring data from Azure applications can be organized into tiers, the highest tiers being your application itself and the lower tiers being components of Azure platform. The method of accessing data from each tier varies. The application tiers are summarized in the table below, and the sources of monitoring data in each tier are presented in the following sections. See Monitoring data locations in Azure for a description of each data location and how you can access its data.
Azure
The following table briefly describes the application tiers that are specific to Azure. Following the link for further details on each in the sections below.
| Tier | Description | Collection method |
|---|---|---|
| Azure Tenant | Data about the operation of tenant-level Azure services, such as Azure Active Directory. | View Azure Active Directory data in portal or configure collection to Azure Monitor using a tenant diagnostic setting. |
| Azure subscription | Data related to the health and management of cross-resource services in your Azure subscription such as Resource Manager and Service Health. | View in portal or configure collection to Azure Monitor using a log profile. |
| Azure resources | Data about the operation and performance of each Azure resource. | Metrics collected automatically, view in Metrics Explorer. Configure diagnostic settings to collect logs in Azure Monitor. Monitoring solutions and Insights available for more detailed monitoring for specific resource types. |
Azure, other cloud, or on-premises
The following table briefly describes the application tiers that may be in Azure, another cloud, or on-premises. Following the link for further details on each in the sections below.
| Tier | Description | Collection method |
|---|---|---|
| Operating system (guest) | Data about the operating system on compute resources. | Install Azure Monitor agent on virtual machines, scale sets and Arc-enabled servers to collect logs and metrics into Azure Monitor. |
| Application Code | Data about the performance and functionality of the actual application and code, including performance traces, application logs, and user telemetry. | Instrument your code to collect data into Application Insights. |
| Custom sources | Data from external services or other components or devices. | Collect log or metrics data into Azure Monitor from any REST client. |
Azure tenant
Telemetry related to your Azure tenant is collected from tenant-wide services such as Azure Active Directory.
Azure Active Directory Audit Logs
Azure Active Directory reporting contains the history of sign-in activity and audit trail of changes made within a particular tenant.
| Destination | Description | Reference |
|---|---|---|
| Azure Monitor Logs | Configure Azure AD logs to be collected in Azure Monitor to analyze them with other monitoring data. | Integrate Azure AD logs with Azure Monitor logs |
| Azure Storage | Export Azure AD logs to Azure Storage for archiving. | Tutorial: Archive Azure AD logs to an Azure storage account |
| Event Hubs | Stream Azure AD logs to other locations using Event Hubs. | Tutorial: Stream Azure Active Directory logs to an Azure event hub. |
Azure subscription
Telemetry related to the health and operation of your Azure subscription.
Azure Activity log
The Azure Activity log includes service health records along with records on any configuration changes made to the resources in your Azure subscription. The Activity log is available to all Azure resources and represents their external view.
| Destination | Description | Reference |
|---|---|---|
| Activity log | The Activity log is collected into its own data store that you can view from the Azure Monitor menu or use to create Activity log alerts. | Query the Activity log with the Azure portal |
| Azure Monitor Logs | Configure Azure Monitor Logs to collect the Activity log to analyze it with other monitoring data. | Collect and analyze Azure activity logs in Log Analytics workspace in Azure Monitor |
| Azure Storage | Export the Activity log to Azure Storage for archiving. | Archive Activity log |
| Event Hubs | Stream the Activity log to other locations using Event Hubs | Stream Activity log to Event Hubs. |
Azure Service Health
Azure Service Health provides information about the health of the Azure services in your subscription that your application and resources rely on.
| Destination | Description | Reference |
|---|---|---|
| Activity log Azure Monitor Logs |
Service Health records are stored in the Azure Activity log, so you can view them in the Azure portal or perform any other activities you can perform with the Activity log. | View service health notifications by using the Azure portal |
Azure Monitor Change Analysis
Change Analysis provides insights into your Azure application changes, increases observability, and reduces mean time to repair.
| Destination | Description | Reference |
|---|---|---|
| Azure Resource Manager control plane changes | Change Analysis provides a historical record of how the Azure resources that host your application have changed over time, using Azure Resource Graph | Resources | Get Changes |
| Resource configurations and settings changes | Change Analysis securely queries and computes IP Configuration rules, TLS settings, and extension versions to provide more change details in the app. | Azure Resource Manager configuration changes |
| Web app in-guest changes | Every 30 minutes, Change Analysis captures the deployment and configuration state of an application. | Diagnose and solve problems tool for Web App |
Azure resources
Metrics and resource logs provide information about the internal operation of Azure resources. These are available for most Azure services, and monitoring solutions and insights collect additional data for particular services.
Platform metrics
Most Azure services will send platform metrics that reflect their performance and operation directly to the metrics database. The specific metrics will vary for each type of resource.
| Destination | Description | Reference |
|---|---|---|
| Azure Monitor Metrics | Platform metrics will write to the Azure Monitor metrics database with no configuration. Access platform metrics from Metrics Explorer. | Getting started with Azure Metrics Explorer Supported metrics with Azure Monitor |
| Azure Monitor Logs | Copy platform metrics to Logs for trending and other analysis using Log Analytics. | Azure diagnostics direct to Log Analytics |
| Azure Monitor Change Analysis | Change Analysis detects various types of changes, from the infrastructure layer through application deployment. | Use Change Analysis in Azure Monitor |
| Event Hubs | Stream metrics to other locations using Event Hubs. | Stream Azure monitoring data to an event hub for consumption by an external tool |
Resource logs
Resource logs provide insights into the internal operation of an Azure resource. Resource logs are created automatically, but you must create a diagnostic setting to specify a destination for them to be collected for each resource.
The configuration requirements and content of resource logs vary by resource type, and not all services yet create them. See Supported services, schemas, and categories for Azure resource logs for details on each service and links to detailed configuration procedures. If the service isn't listed in this article, then that service doesn't currently create resource logs.
| Destination | Description | Reference |
|---|---|---|
| Azure Monitor Logs | Send resource logs to Azure Monitor Logs for analysis with other collected log data. | Collect Azure resource logs in Log Analytics workspace in Azure Monitor |
| Storage | Send resource logs to Azure Storage for archiving. | Archive Azure resource logs |
| Event Hubs | Stream resource logs to other locations using Event Hubs. | Stream Azure resource logs to an event hub |
Operating system (guest)
Compute resources in Azure, in other clouds, and on-premises have a guest operating system to monitor. With the installation of an agent, you can gather telemetry from the guest into Azure Monitor to analyze it with the same monitoring tools as the Azure services themselves.
Azure Monitor agent
Install the Azure Monitor agent for comprehensive monitoring and management of your Windows or Linux virtual machines, scale sets and Arc-enabled servers. The Azure Monitor agent replaces the Log Analytics agent and Azure diagnostic extension.
| Destination | Description | Reference |
|---|---|---|
| Azure Monitor Logs | The Azure Monitor agent allows you to collect logs from data sources that you configure using data collection rules or from monitoring solutions that provide additional insights into applications running on the machine. These can be sent to one or more Log Analytics workspaces. | Data sources and destinations |
| Azure Monitor Metrics (preview) | The Azure Monitor agent allows you to collect performance counters and send them to Azure Monitor metrics database | Data sources and destinations |
Log Analytics agent
Install the Log Analytics agent for comprehensive monitoring and management of your Windows or Linux virtual machines. The virtual machine can be running in Azure, another cloud, or on-premises. The Log Analytics agent is still supported but has been replaced by the Azure Monitor agent.
| Destination | Description | Reference |
|---|---|---|
| Azure Monitor Logs | The Log Analytics agent connects to Azure Monitor either directly or through System Center Operations Manager and allows you to collect data from data sources that you configure or from monitoring solutions that provide additional insights into applications running on the virtual machine. | Agent data sources in Azure Monitor Connect Operations Manager to Azure Monitor |
Azure diagnostic extension
Enabling the Azure diagnostics extension for Azure Virtual machines allows you to collect logs and metrics from the guest operating system of Azure compute resources including Azure Cloud Service (classic) Web and Worker Roles, Virtual Machines, Virtual Machine Scale Sets, and Service Fabric.
| Destination | Description | Reference |
|---|---|---|
| Storage | Azure diagnostics extension always writes to an Azure Storage account. | Install and configure Azure diagnostics extension (WAD) Use Linux Diagnostic Extension to monitor metrics and logs |
| Azure Monitor Metrics (preview) | When you configure the Diagnostics Extension to collect performance counters, they are written to the Azure Monitor metrics database. | Send Guest OS metrics to the Azure Monitor metric store using a Resource Manager template for a Windows virtual machine |
| Event Hubs | Configure the Diagnostics Extension to stream the data to other locations using Event Hubs. | Streaming Azure Diagnostics data by using Event Hubs Use Linux Diagnostic Extension to monitor metrics and logs |
| Application Insights Logs | Collect logs and performance counters from the compute resource supporting your application to be analyzed with other application data. | Send Cloud Service, Virtual Machine, or Service Fabric diagnostic data to Application Insights |
VM insights
VM insights provides a customized monitoring experience for virtual machines providing features beyond core Azure Monitor functionality. It requires a Dependency Agent on Windows and Linux virtual machines that integrates with the Log Analytics agent to collect discovered data about processes running on the virtual machine and external process dependencies.
| Destination | Description | Reference |
|---|---|---|
| Azure Monitor Logs | Stores data about processes and dependencies on the agent. | Using VM insights Map to understand application components |
Application Code
Detailed application monitoring in Azure Monitor is done with Application Insights, which collects data from applications running on various platforms. The application can be running in Azure, another cloud, or on-premises.
Application data
When you enable Application Insights for an application by installing an instrumentation package, it collects metrics and logs related to the performance and operation of the application. Application Insights stores the data it collects in the same Azure Monitor data platform used by other data sources. It includes extensive tools for analyzing this data, but you can also analyze it with data from other sources using tools such as Metrics Explorer, Log Analytics, and Change Analysis.
| Destination | Description | Reference |
|---|---|---|
| Azure Monitor Logs | Operational data about your application including page views, application requests, exceptions, and traces. | Analyze log data in Azure Monitor |
| Dependency information between application components to support Application Map and telemetry correlation. | Telemetry correlation in Application Insights Application Map |
|
| Results of availability tests that test the availability and responsiveness of your application from different locations on the public Internet. | Monitor availability and responsiveness of any web site | |
| Azure Monitor Metrics | Application Insights collects metrics describing the performance and operation of the application in addition to custom metrics that you define in your application into the Azure Monitor metrics database. | Log-based and pre-aggregated metrics in Application Insights Application Insights API for custom events and metrics |
| Azure Monitor Change Analysis | Change Analysis detects and provides insights on various types of changes in your application. | Use Change Analysis in Azure Monitor |
| Azure Storage | Send application data to Azure Storage for archiving. | Export telemetry from Application Insights |
| Details of availability tests are stored in Azure Storage. Use Application Insights in the Azure portal to download for local analysis. Results of availability tests are stored in Azure Monitor Logs. | Monitor availability and responsiveness of any web site | |
| Profiler trace data is stored in Azure Storage. Use Application Insights in the Azure portal to download for local analysis. | Profile production applications in Azure with Application Insights | |
| Debug snapshot data that is captured for a subset of exceptions is stored in Azure Storage. Use Application Insights in the Azure portal to download for local analysis. | How snapshots work |
Insights
Insights collect data to provide additional insights into the operation of a particular service or application. They may address resources in different application tiers and even multiple tiers.
Container insights
Container insights provides a customized monitoring experience for Azure Kubernetes Service (AKS). It collects additional data about these resources described in the following table.
| Destination | Description | Reference |
|---|---|---|
| Azure Monitor Logs | Stores monitoring data for AKS including inventory, logs, and events. Metric data is also stored in Logs in order to leverage its analysis functionality in the portal. | Understand AKS cluster performance with Container insights |
| Azure Monitor Metrics | Metric data is stored in the metric database to drive visualization and alerts. | View container metrics in metrics explorer |
| Azure Kubernetes Service | Provides direct access to your Azure Kubernetes Service (AKS) container logs (stdout/stderror), events, and pod metrics in the portal. | How to view Kubernetes logs, events, and pod metrics in real-time |
VM insights
VM insights provides a customized experience for monitoring virtual machines. A description of the data collected by VM insights is included in the Operating System (guest) section above.
Custom sources
In addition to the standard tiers of an application, you may need to monitor other resources that have telemetry that can't be collected with the other data sources. For these resources, write this data to either Metrics or Logs using an Azure Monitor API.
| Destination | Method | Description | Reference |
|---|---|---|---|
| Azure Monitor Logs | Logs ingestion API | Collect log data from any REST client and store in Log Analytics workspace using a data collection rule. | Logs ingestion API in Azure Monitor (preview) |
| Data Collector API | Collect log data from any REST client and store in Log Analytics workspace. | Send log data to Azure Monitor with the HTTP Data Collector API (preview) | |
| Azure Monitor Metrics | Custom Metrics API | Collect metric data from any REST client and store in Azure Monitor metrics database. | Send custom metrics for an Azure resource to the Azure Monitor metric store by using a REST API |
Other services
Other services in Azure write data to the Azure Monitor data platform. This allows you to analyze data collected by these services with data collected by Azure Monitor and apply the same analysis and visualization tools.
| Service | Destination | Description | Reference |
|---|---|---|---|
| Microsoft Defender for Cloud | Azure Monitor Logs | Microsoft Defender for Cloud stores the security data it collects in a Log Analytics workspace, which allows it to be analyzed with other log data collected by Azure Monitor. | Data collection in Microsoft Defender for Cloud |
| Microsoft Sentinel | Azure Monitor Logs | Microsoft Sentinel stores the data it collects from different data sources in a Log Analytics workspace, which allows it to be analyzed with other log data collected by Azure Monitor. | Connect data sources |
Next steps
- Learn more about the types of monitoring data collected by Azure Monitor and how to view and analyze this data.
- List the different locations where Azure resources store data and how you can access it.
Feedback
Submit and view feedback for