Synapse implementation success methodology: Perform monitoring review
Note
This article forms part of the Azure Synapse implementation success by design series of articles. For an overview of the series, see Azure Synapse implementation success by design.
Monitoring is a key part of the operationalization of any Azure solution. This article provides guidance on reviewing and configuring the monitoring of your Azure Synapse Analytics environment. Key to this activity is the identification of what needs to be monitored and who needs to review the monitoring results.
Using your solution requirements and other data collected during the assessment stage and solution development, build a list of important behaviors and activities that need to be monitored in your production environment. As you build this list, identify the groups of users that will need access to monitoring information and build the procedures they can follow to respond to monitoring results.
You can use Azure Monitor to provide base-level infrastructure metrics, alerts, and logs for most Azure services. Azure diagnostic logs are emitted by a resource to provide rich, frequent data about the operation of that resource. Azure Synapse can write diagnostic logs in Azure Monitor.
For more information, see Use Azure Monitor with your Azure Synapse Analytics workspace.
Monitor dedicated SQL pools
You can monitor a dedicated SQL pool by using Azure Monitor, alerting, dynamic management views (DMVs), and Log Analytics.
- Alerts: You can set up alerts that send you an email or call a webhook when a certain metric reaches a predefined threshold. For example, you can receive an alert email when the database size grows too large. For more information, see Alerts.
- DMVs: You can use DMVs to monitor workloads to help investigate query executions in SQL pools.
- Log Analytics: Log Analytics is a tool in the Azure portal that you can use to edit and run log queries from data collected by Azure Monitor. For more information, see Monitor workload - Azure portal.
Monitor serverless SQL pools
You can monitor a serverless SQL pool by monitoring your SQL requests in Synapse Studio. That way, you can keep an eye on the status of running requests and review details of historical requests.
Monitor Spark pools
You can monitor your Apache Spark applications in Synapse Studio. That way, you can keep an eye on the latest status, issues, and progress.
You can enable the Synapse Studio connector that's built in to Log Analytics. You can then collect and send Apache Spark application metrics and logs to your Log Analytics workspace. You can also use an Azure Monitor workbook to visualize the metrics and logs. For more information, see Monitor Apache Spark applications with Azure Log Analytics.
Monitor pipelines
Azure Synapse allows you to create complex pipelines that automate and integrate your data movement, data transformation, and compute activities. You can author and monitor pipelines by using Synapse Studio to keep an eye on the latest status, issues, and progress of your pipelines. For more information, see Use Synapse Studio to monitor your workspace pipeline runs.
Next steps
For more information about this article, check out the following resources: