ADF Pipeline monitoring template using log analytics

Abhiram Duvvuru 171 Reputation points Microsoft Employee
2023-06-10T03:09:10.01+00:00

Hi Team,

I want to build dashboard to track health of ADF pipelines using log analytics i.e (SADFPipelineRuns, ActvitiyRuns,TriggerRuns). I'm looking for templates for reference such as Power BI/ Grafana or any other . Please share if there are any such kind of templates.

Thanks,

Abhiram

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,643 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. QuantumCache 20,031 Reputation points
    2023-06-10T15:20:11.25+00:00

    Hello @Abhiram Duvvuru
    How are you currently monitoring the ADF activities ?

    Did you try to configure diagnostic settings in ADF to send data to Log Analytics,?

    there are several templates available for building dashboards to track the health of Azure Data Factory pipelines using Log Analytics. Here are a few options:

    Azure Monitor Workbook: Azure Monitor provides a built-in workbook for Azure Data Factory that allows you to monitor the health of your pipelines, datasets, and activities. The workbook provides several pre-built visualizations and allows you to customize the dashboard to meet your specific needs.

    Power BI template: Microsoft provides a Power BI template for Azure Data Factory that allows you to monitor the health of your pipelines, datasets, and activities. The template provides several pre-built visualizations and allows you to customize the dashboard to meet your specific needs.

    Grafana dashboard: Grafana is an open-source platform for building dashboards and visualizations. There are several community-built Grafana dashboards available for monitoring Azure Data Factory pipelines using Log Analytics. You can find these dashboards on the Grafana website or in the Grafana Marketplace.

    Custom dashboard: You can also build a custom dashboard using Log Analytics queries and visualizations. Log Analytics provides several built-in visualizations, such as line charts, bar charts, and tables, that you can use to display your data. You can also use custom visualizations, such as those provided by Power BI or Grafana, to display your data.