Edit

Cost optimization in Azure Monitor

  • Article

Cost optimization refers to ways to reduce unnecessary expenses and improve operational efficiencies. You can significantly reduce your cost for Azure Monitor by understanding your different configuration options and opportunities to reduce the amount of data that it collects. Before you use this article, you should see Azure Monitor cost and usage to understand the different ways that Azure Monitor charges and how to view your monthly bill.

This article describes Cost optimization for Azure Monitor as part of the Azure Well-Architected Framework. This is a set of guiding tenets that can be used to improve the quality of a workload. The framework consists of five pillars of architectural excellence:

  • Reliability
  • Security
  • Cost Optimization
  • Operational Excellence
  • Performance Efficiency

Azure Monitor Logs

Design checklist

  • Configure pricing tier for the amount of data that each Log Analytics workspace typically collects.
  • Configure tables used for debugging, troubleshooting, and auditing as Basic Logs.
  • Configure data retention and archiving.
  • Regularly analyze collected data to identify trends and anomalies.
  • Create an alert when data collection is high.
  • Consider a daily cap as a preventative measure to ensure that you don't exceed a particular budget.

Configuration recommendations

Recommendation Benefit
Determine whether to combine your operational data and your security data in the same Log Analytics workspace. Since all data in a Log Analytics workspace is subject to Microsoft Sentinel pricing if Sentinel is enabled, there may be cost implications to combining this data. See Design a Log Analytics workspace architecture for details on making this decision for your environment balancing it with criteria in other pillars.
Configure pricing tier for the amount of data that each Log Analytics workspace typically collects. By default, Log Analytics workspaces will use pay-as-you-go pricing with no minimum data volume. If you collect enough data, you can significantly decrease your cost by using a commitment tier, which allows you to commit to a daily minimum of data collected in exchange for a lower rate. If you collect enough data across workspaces in a single region, you can link them to a dedicated cluster and combine their collected volume using cluster pricing.

See Azure Monitor Logs cost calculations and options for details on commitment tiers and guidance on determining which is most appropriate for your level of usage. See Usage and estimated costs to view estimated costs for your usage at different pricing tiers.
Configure data retention and archiving. There is a charge for retaining data in a Log Analytics workspace beyond the default of 31 days (90 days if Sentinel is enabled on the workspace and 90 days for Application insights data). Consider your particular requirements for having data readily available for log queries. You can significantly reduce your cost by configuring Archived Logs, which allows you to retain data for up to seven years and still access it occasionally using search jobs or restoring a set of data to the workspace.
Configure tables used for debugging, troubleshooting, and auditing as Basic Logs. Tables in a Log Analytics workspace configured for Basic Logs have a lower ingestion cost in exchange for limited features and a charge for log queries. If you query these tables infrequently and don't use them for alerting, this query cost can be more than offset by the reduced ingestion cost.
Regularly analyze collected data to identify trends and anomalies. Use Log Analytics workspace insights to periodically review the amount of data collected in your workspace. In addition to helping you understand the amount of data collected by different sources, it will identify anomalies and upward trends in data collection that could result in excess cost. Further analyze data collection using methods in Analyze usage in Log Analytics workspace to determine if there's additional configuration that can decrease your usage further. This is particularly important when you add a new set of data sources, such as a new set of virtual machines or onboard a new service.
Create an alert when data collection is high. To avoid unexpected bills, you should be proactively notified anytime you experience excessive usage. Notification allows you to address any potential anomalies before the end of your billing period.
Consider a daily cap as a preventative measure to ensure that you don't exceed a particular budget. A daily cap disables data collection in a Log Analytics workspace for the rest of the day after your configured limit is reached. This shouldn't be used as a method to reduce costs as described in When to use a daily cap.

If you do set a daily cap, in addition to creating an alert when the cap is reached,ensure that you also create an alert rule to be notified when some percentage has been reached (90% for example). This gives you an opportunity to investigate and address the cause of the increased data before the cap shuts off data collection.

Azure resources

Design checklist

  • Collect only critical resource log data from Azure resources.

Configuration recommendations

Recommendation Benefit
Collect only critical resource log data from Azure resources. When you create diagnostic settings to send resource logs for your Azure resources to a Log Analytics database, only specify those categories that you require. Since diagnostic settings don't allow granular filtering of resource logs, you can use a workspace transformation to further filter unneeded data for those resources that use a supported table. See Diagnostic settings in Azure Monitor for details on how to configure diagnostic settings and using transformations to filter their data.

Virtual machines

Design checklist

  • Migrate from Log Analytics agent to Azure Monitor agent for granular data filtering.
  • Filter data that you don't require from agents.
  • Determine whether you'll use VM insights and what data to collect.
  • Reduce polling frequency of performance counters.
  • Ensure that VMs aren't sending duplicate data.
  • Use Log Analytics workspace insights to analyze billable costs and identify cost saving opportunities.
  • Migrate your SCOM environment to Azure Monitor SCOM Managed Instance.

Configuration recommendations

Recommendation Description
Migrate from Log Analytics agent to Azure Monitor agent for granular data filtering. If you still have VMs with the Log Analytics agent, migrate them to Azure Monitor agent so you can take advantage of better data filtering and use unique configurations with different sets of VMs. Configuration for data collection by the Log Analytics agent is done on the workspace, so all agents receive the same configuration. Data collection rules used by Azure Monitor agent can be tuned to the specific monitoring requirements of different sets of VMs. The Azure Monitor agent also allows you to use transformations to filter data being collected.
Filter data that you don't require from agents. Reduce your data ingestion costs by filtering data that you don't use for alerting or analysis. See Monitor virtual machines with Azure Monitor: Collect data for guidance on data to collect for different monitoring scenarios and Control costs for specific guidance on filtering data to reduce your costs.
Determine what data to collect with VM insights. VM insights is a great feature to quickly get started with monitoring your VMs and provides powerful features such as Map and performance trend views. If you don't use the Map feature or the data that it collects, then you should disable collection of processes and dependency data in your VM insights configuration to save on data ingestion costs.
Reduce polling frequency of performance counters. If you're using a data collection rule to send performance data to your Log Analytics workspace, you can reduce their polling frequency to reduce the amount of data collected.
Ensure that VMs aren't sending duplicate data. If you multi-home agents or you create similar data collection rules, make sure you're sending unique data to each workspace. See Analyze usage in Log Analytics workspace for guidance on analyzing your collected data to make sure you aren't collecting duplicate data. If you're migrating between agents, continue to use the Log Analytics agent until you migrate to the Azure Monitor agent rather than using both together unless you can ensure that each is collecting unique data.
Use Log Analytics workspace insights to analyze billable costs and identify cost saving opportunities. Log Analytics workspace insights shows you the billable data collected in each table and from each VM. Use this information to identify your top machines and tables since they represent your best opportunity to reduce costs by filtering data. Use this insight and log queries in Analyze usage in Log Analytics workspace to further analyze the effects of configuration changes.
Migrate your SCOM environment to Azure Monitor SCOM Managed Instance. Migrate your existing SCOM environment to Azure Monitor SCOM Managed Instance to support any management packs that can't be replaced by Azure Monitor. SCOM managed instance removes the requirement to maintain local management servers and database servers, reducing your overall cost to maintain your SCOM infrastructure.

Container insights

Design checklist

  • Configure agent collection to remove unneeded data.
  • Modify settings for collection of metric data.
  • Limit Prometheus metrics collected.
  • Configure Basic Logs.

Configuration recommendations

Recommendation Benefit
Configure agent collection to remove unneeded data. Analyze the data collected by Container insights as described in Controlling ingestion to reduce cost and adjust your configuration to stop collection of data in ContainerLogs you don't need.
Modify settings for collection of metric data. You can reduce your costs by modifying the default collection settings Container insights uses for the collection of metric data. See Enable cost optimization settings (preview) for details on modifying both the frequency that metric data is collected and the namespaces that are collected.
Limit Prometheus metrics collected. If you configured Prometheus metric scraping, then follow the recommendations at Controlling ingestion to reduce cost to optimize your data collection for cost.
Configure Basic Logs. Convert your schema to ContainerLogV2 which is compatible with Basic logs and can provide significant cost savings as described in Controlling ingestion to reduce cost.

Application Insights

Design checklist

  • Use sampling to tune the amount of data collected.
  • Use sampling to tune the amount of data collected.
  • Limit the number of Ajax calls.
  • Disable unneeded modules.
  • Pre-aggregate metrics from any calls to TrackMetric.
  • Limit the use of custom metrics.
  • Ensure use of updated SDKs.

Configuration recommendations

Recommendation Benefit
Change to Workspace-based Application Insights Ensure that your Application Insights resources are Workspace-based so that they can leverage new cost savings tools such as Basic Logs, Commitment Tiers, Retention by data type and Data Archive.
Use sampling to tune the amount of data collected. Sampling is the primary tool you can use to tune the amount of data collected by Application Insights. Use sampling to reduce the amount of telemetry that's sent from your applications with minimal distortion of metrics.
Limit the number of Ajax calls. Limit the number of Ajax calls that can be reported in every page view or disable Ajax reporting. If you disable Ajax calls, you'll be disabling JavaScript correlation too.
Disable unneeded modules. Edit ApplicationInsights.config to turn off collection modules that you don't need. For example, you might decide that performance counters or dependency data aren't required.
Pre-aggregate metrics from any calls to TrackMetric. If you put calls to TrackMetric in your application, you can reduce traffic by using the overload that accepts your calculation of the average and standard deviation of a batch of measurements. Alternatively, you can use a pre-aggregating package.
Limit the use of custom metrics. The Application Insights option to Enable alerting on custom metric dimensions can increase costs. Using this option can result in the creation of more pre-aggregation metrics.
Ensure use of updated SDKs. Earlier versions of the ASP.NET Core SDK and Worker Service SDK collect many counters by default, which were collected as custom metrics. Use later versions to specify only required counters.

Next step