Azure Data Factory monitoring data reference

This article contains all the monitoring reference information for this service.

See Monitor Data Factory for details on the data you can collect for Azure Data Factory and how to use it.

Metrics

This section lists all the automatically collected platform metrics for this service. These metrics are also part of the global list of all platform metrics supported in Azure Monitor.

For information on metric retention, see Azure Monitor Metrics overview.

Supported metrics for Microsoft.DataFactory/datafactories

The following table lists the metrics available for the Microsoft.DataFactory/datafactories resource type.

• All columns might not be present in every table.
• Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.

Table headings

• Category - The metrics group or classification.
• Metric - The metric display name as it appears in the Azure portal.
• Name in REST API - The metric name as referred to in the REST API.
• Unit - Unit of measure.
• Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
• Dimensions - Dimensions available for the metric.
• Time Grains - Intervals at which the metric is sampled. For example, PT1M indicates that the metric is sampled every minute, PT30M every 30 minutes, PT1H every hour, and so on.
• DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.

Metric	Name in REST API	Unit	Aggregation	Dimensions	Time Grains	DS Export
Failed Runs Failed Runs	FailedRuns	Count	Total	pipelineName, activityName	PT1H	Yes
Successful Runs Successful Runs	SuccessfulRuns	Count	Total	pipelineName, activityName	PT1H	Yes

Supported metrics for Microsoft.DataFactory/factories

The following table lists the metrics available for the Microsoft.DataFactory/factories resource type.

• All columns might not be present in every table.
• Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.

Table headings

• Category - The metrics group or classification.
• Metric - The metric display name as it appears in the Azure portal.
• Name in REST API - The metric name as referred to in the REST API.
• Unit - Unit of measure.
• Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
• Dimensions - Dimensions available for the metric.
• Time Grains - Intervals at which the metric is sampled. For example, PT1M indicates that the metric is sampled every minute, PT30M every 30 minutes, PT1H every hour, and so on.
• DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.

Metric	Name in REST API	Unit	Aggregation	Dimensions	Time Grains	DS Export
Cancelled activity runs metrics Cancelled activity runs metrics	ActivityCancelledRuns	Count	Total	ActivityType, PipelineName, FailureType, Name	PT1M	Yes
Failed activity runs metrics Failed activity runs metrics	ActivityFailedRuns	Count	Total	ActivityType, PipelineName, FailureType, Name	PT1M	Yes
Succeeded activity runs metrics Succeeded activity runs metrics	ActivitySucceededRuns	Count	Total	ActivityType, PipelineName, FailureType, Name	PT1M	Yes
Airflow Integration Runtime Celery Task Timeout Error Airflow Integration Runtime Celery Task Timeout Error	AirflowIntegrationRuntimeCeleryTaskTimeoutError	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Collect DB Dags Airflow Integration Runtime Collect DB Dags	AirflowIntegrationRuntimeCollectDBDags	Milliseconds	Average	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Cpu Percentage Airflow Integration Runtime Cpu Percentage	AirflowIntegrationRuntimeCpuPercentage	Percent	Average	IntegrationRuntimeName, ContainerName	PT1M	Yes
Airflow Integration Runtime Cpu Usage Airflow Integration Runtime Cpu Usage	AirflowIntegrationRuntimeCpuUsage	Millicores	Average	IntegrationRuntimeName, ContainerName	PT1M	Yes
Airflow Integration Runtime Dag Bag Size Airflow Integration Runtime Dag Bag Size	AirflowIntegrationRuntimeDagBagSize	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Dag Callback Exceptions Airflow Integration Runtime Dag Callback Exceptions	AirflowIntegrationRuntimeDagCallbackExceptions	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime DAG File Refresh Error Airflow Integration Runtime DAG File Refresh Error	AirflowIntegrationRuntimeDAGFileRefreshError	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime DAG Processing Import Errors Airflow Integration Runtime DAG Processing Import Errors	AirflowIntegrationRuntimeDAGProcessingImportErrors	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime DAG Processing Last Duration Airflow Integration Runtime DAG Processing Last Duration	AirflowIntegrationRuntimeDAGProcessingLastDuration	Milliseconds	Average	IntegrationRuntimeName, DagFile	PT1M	No
Airflow Integration Runtime DAG Processing Last Run Seconds Ago Airflow Integration Runtime DAG Processing Last Run Seconds Ago	AirflowIntegrationRuntimeDAGProcessingLastRunSecondsAgo	Seconds	Average	IntegrationRuntimeName, DagFile	PT1M	No
Airflow Integration Runtime DAG ProcessingManager Stalls Airflow Integration Runtime DAG ProcessingManager Stalls	AirflowIntegrationRuntimeDAGProcessingManagerStalls	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime DAG Processing Processes Airflow Integration Runtime DAG Processing Processes	AirflowIntegrationRuntimeDAGProcessingProcesses	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime DAG Processing Processor Timeouts Airflow Integration Runtime DAG Processing Processor Timeouts	AirflowIntegrationRuntimeDAGProcessingProcessorTimeouts	Seconds	Average	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime DAG Processing Total Parse Time Airflow Integration Runtime DAG Processing Total Parse Time	AirflowIntegrationRuntimeDAGProcessingTotalParseTime	Seconds	Average	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime DAG Run Dependency Check Airflow Integration Runtime DAG Run Dependency Check	AirflowIntegrationRuntimeDAGRunDependencyCheck	Milliseconds	Average	IntegrationRuntimeName, DagId	PT1M	No
Airflow Integration Runtime DAG Run Duration Failed Airflow Integration Runtime DAG Run Duration Failed	AirflowIntegrationRuntimeDAGRunDurationFailed	Milliseconds	Average	IntegrationRuntimeName, DagId	PT1M	No
Airflow Integration Runtime DAG Run Duration Success Airflow Integration Runtime DAG Run Duration Success	AirflowIntegrationRuntimeDAGRunDurationSuccess	Milliseconds	Average	IntegrationRuntimeName, DagId	PT1M	No
Airflow Integration Runtime DAG Run First Task Scheduling Delay Airflow Integration Runtime DAG Run First Task Scheduling Delay	AirflowIntegrationRuntimeDAGRunFirstTaskSchedulingDelay	Milliseconds	Average	IntegrationRuntimeName, DagId	PT1M	No
Airflow Integration Runtime DAG Run Schedule Delay Airflow Integration Runtime DAG Run Schedule Delay	AirflowIntegrationRuntimeDAGRunScheduleDelay	Milliseconds	Average	IntegrationRuntimeName, DagId	PT1M	No
Airflow Integration Runtime Executor Open Slots Airflow Integration Runtime Executor Open Slots	AirflowIntegrationRuntimeExecutorOpenSlots	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Executor Queued Tasks Airflow Integration Runtime Executor Queued Tasks	AirflowIntegrationRuntimeExecutorQueuedTasks	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Executor Running Tasks Airflow Integration Runtime Executor Running Tasks	AirflowIntegrationRuntimeExecutorRunningTasks	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Job End Airflow Integration Runtime Job End	AirflowIntegrationRuntimeJobEnd	Count	Total	IntegrationRuntimeName, Job	PT1M	No
Airflow Integration Runtime Heartbeat Failure Airflow Integration Runtime Heartbeat Failure	AirflowIntegrationRuntimeJobHeartbeatFailure	Count	Total	IntegrationRuntimeName, Job	PT1M	No
Airflow Integration Runtime Job Start Airflow Integration Runtime Job Start	AirflowIntegrationRuntimeJobStart	Count	Total	IntegrationRuntimeName, Job	PT1M	No
Airflow Integration Runtime Memory Percentage Airflow Integration Runtime Memory Percentage	AirflowIntegrationRuntimeMemoryPercentage	Percent	Average	IntegrationRuntimeName, ContainerName	PT1M	Yes
Airflow Integration Runtime Memory Usage Airflow Integration Runtime Memory Usage	AirflowIntegrationRuntimeMemoryUsage	Bytes	Average	IntegrationRuntimeName, ContainerName	PT1M	Yes
Airflow Integration Runtime Node Count Airflow Integration Runtime Node Count	AirflowIntegrationRuntimeNodeCount	Count	Average	IntegrationRuntimeName, ComputeNodeSize	PT1M	Yes
Airflow Integration Runtime Operator Failures Airflow Integration Runtime Operator Failures	AirflowIntegrationRuntimeOperatorFailures	Count	Total	IntegrationRuntimeName, Operator	PT1M	No
Airflow Integration Runtime Operator Successes Airflow Integration Runtime Operator Successes	AirflowIntegrationRuntimeOperatorSuccesses	Count	Total	IntegrationRuntimeName, Operator	PT1M	No
Airflow Integration Runtime Pool Open Slots Airflow Integration Runtime Pool Open Slots	AirflowIntegrationRuntimePoolOpenSlots	Count	Total	IntegrationRuntimeName, Pool	PT1M	No
Airflow Integration Runtime Pool Queued Slots Airflow Integration Runtime Pool Queued Slots	AirflowIntegrationRuntimePoolQueuedSlots	Count	Total	IntegrationRuntimeName, Pool	PT1M	No
Airflow Integration Runtime Pool Running Slots Airflow Integration Runtime Pool Running Slots	AirflowIntegrationRuntimePoolRunningSlots	Count	Total	IntegrationRuntimeName, Pool	PT1M	No
Airflow Integration Runtime Pool Starving Tasks Airflow Integration Runtime Pool Starving Tasks	AirflowIntegrationRuntimePoolStarvingTasks	Count	Total	IntegrationRuntimeName, Pool	PT1M	No
Airflow Integration Runtime Scheduler Critical Section Busy Airflow Integration Runtime Scheduler Critical Section Busy	AirflowIntegrationRuntimeSchedulerCriticalSectionBusy	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Scheduler Critical Section Duration Airflow Integration Runtime Scheduler Critical Section Duration	AirflowIntegrationRuntimeSchedulerCriticalSectionDuration	Milliseconds	Average	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Scheduler Failed SLA Email Attempts Airflow Integration Runtime Scheduler Failed SLA Email Attempts	AirflowIntegrationRuntimeSchedulerFailedSLAEmailAttempts	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Scheduler Heartbeats Airflow Integration Runtime Scheduler Heartbeats	AirflowIntegrationRuntimeSchedulerHeartbeat	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Scheduler Orphaned Tasks Adopted Airflow Integration Runtime Scheduler Orphaned Tasks Adopted	AirflowIntegrationRuntimeSchedulerOrphanedTasksAdopted	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Scheduler Orphaned Tasks Cleared Airflow Integration Runtime Scheduler Orphaned Tasks Cleared	AirflowIntegrationRuntimeSchedulerOrphanedTasksCleared	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Scheduler Tasks Executable Airflow Integration Runtime Scheduler Tasks Executable	AirflowIntegrationRuntimeSchedulerTasksExecutable	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Scheduler Tasks Killed Externally Airflow Integration Runtime Scheduler Tasks Killed Externally	AirflowIntegrationRuntimeSchedulerTasksKilledExternally	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Scheduler Tasks Running Airflow Integration Runtime Scheduler Tasks Running	AirflowIntegrationRuntimeSchedulerTasksRunning	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Scheduler Tasks Starving Airflow Integration Runtime Scheduler Tasks Starving	AirflowIntegrationRuntimeSchedulerTasksStarving	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Started Task Instances Airflow Integration Runtime Started Task Instances	AirflowIntegrationRuntimeStartedTaskInstances	Count	Total	IntegrationRuntimeName, DagId, TaskId	PT1M	No
Airflow Integration Runtime Task Instance Created Using Operator Airflow Integration Runtime Task Instance Created Using Operator	AirflowIntegrationRuntimeTaskInstanceCreatedUsingOperator	Count	Total	IntegrationRuntimeName, Operator	PT1M	No
Airflow Integration Runtime Task Instance Duration Airflow Integration Runtime Task Instance Duration	AirflowIntegrationRuntimeTaskInstanceDuration	Milliseconds	Average	IntegrationRuntimeName, DagId, TaskID	PT1M	No
Airflow Integration Runtime Task Instance Failures Airflow Integration Runtime Task Instance Failures	AirflowIntegrationRuntimeTaskInstanceFailures	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Task Instance Finished Airflow Integration Runtime Task Instance Finished	AirflowIntegrationRuntimeTaskInstanceFinished	Count	Total	IntegrationRuntimeName, DagId, TaskId, State	PT1M	No
Airflow Integration Runtime Task Instance Previously Succeeded Airflow Integration Runtime Task Instance Previously Succeeded	AirflowIntegrationRuntimeTaskInstancePreviouslySucceeded	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Task Instance Successes Airflow Integration Runtime Task Instance Successes	AirflowIntegrationRuntimeTaskInstanceSuccesses	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Task Removed From DAG Airflow Integration Runtime Task Removed From DAG	AirflowIntegrationRuntimeTaskRemovedFromDAG	Count	Total	IntegrationRuntimeName, DagId	PT1M	No
Airflow Integration Runtime Task Restored To DAG Airflow Integration Runtime Task Restored To DAG	AirflowIntegrationRuntimeTaskRestoredToDAG	Count	Total	IntegrationRuntimeName, DagId	PT1M	No
Airflow Integration Runtime Triggers Blocked Main Thread Airflow Integration Runtime Triggers Blocked Main Thread	AirflowIntegrationRuntimeTriggersBlockedMainThread	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Triggers Failed Airflow Integration Runtime Triggers Failed	AirflowIntegrationRuntimeTriggersFailed	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Triggers Running Airflow Integration Runtime Triggers Running	AirflowIntegrationRuntimeTriggersRunning	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Triggers Succeeded Airflow Integration Runtime Triggers Succeeded	AirflowIntegrationRuntimeTriggersSucceeded	Count	Total	IntegrationRuntimeName	PT1M	No
Airflow Integration Runtime Zombie Tasks Killed Airflow Integration Runtime Zombie Tasks Killed	AirflowIntegrationRuntimeZombiesKilled	Count	Total	IntegrationRuntimeName	PT1M	No
Total factory size (GB unit) Total factory size (GB unit)	FactorySizeInGbUnits	Count	Maximum	<none>	PT1M	Yes
Integration runtime available memory Integration runtime available memory	IntegrationRuntimeAvailableMemory	Bytes	Average	IntegrationRuntimeName, NodeName	PT1M	Yes
Integration runtime available node count Integration runtime available node count	IntegrationRuntimeAvailableNodeNumber	Count	Average	IntegrationRuntimeName	PT1M	Yes
Integration runtime queue duration Integration runtime queue duration	IntegrationRuntimeAverageTaskPickupDelay	Seconds	Average	IntegrationRuntimeName	PT1M	Yes
Integration runtime CPU utilization Integration runtime CPU utilization	IntegrationRuntimeCpuPercentage	Percent	Average	IntegrationRuntimeName, NodeName	PT1M	Yes
Integration runtime queue length Integration runtime queue length	IntegrationRuntimeQueueLength	Count	Average	IntegrationRuntimeName	PT1M	Yes
Maximum allowed factory size (GB unit) Maximum allowed factory size (GB unit)	MaxAllowedFactorySizeInGbUnits	Count	Maximum	<none>	PT1M	Yes
Maximum allowed entities count Maximum allowed entities count	MaxAllowedResourceCount	Count	Maximum	<none>	PT1M	Yes
Copy available capacity percentage of MVNet integration runtime Copy available capacity percentage of MVNet integration runtime	MVNetIRCopyAvailableCapacityPCT	Percent	Maximum	IntegrationRuntimeName	PT1M	Yes
Copy capacity utilization of MVNet integration runtime Copy capacity utilization of MVNet integration runtime	MVNetIRCopyCapacityUtilization	Percent	Maximum	IntegrationRuntimeName	PT1M	Yes
Copy waiting queue length of MVNet integration runtime Copy waiting queue length of MVNet integration runtime	MVNetIRCopyWaitingQueueLength	Count	Average	IntegrationRuntimeName	PT1M	Yes
External available capacity percentage of MVNet integration runtime External available capacity percentage of MVNet integration runtime	MVNetIRExternalAvailableCapacityPCT	Percent	Maximum	IntegrationRuntimeName	PT1M	Yes
External capacity utilization of MVNet integration runtime External capacity utilization of MVNet integration runtime	MVNetIRExternalCapacityUtilization	Percent	Maximum	IntegrationRuntimeName	PT1M	Yes
External waiting queue length of MVNet integration runtime External waiting queue length of MVNet integration runtime	MVNetIRExternalWaitingQueueLength	Count	Average	IntegrationRuntimeName	PT1M	Yes
Pipeline available capacity percentage of MVNet integration runtime Pipeline available capacity percentage of MVNet integration runtime	MVNetIRPipelineAvailableCapacityPCT	Percent	Maximum	IntegrationRuntimeName	PT1M	Yes
Pipeline capacity utilization of MVNet integration runtime Pipeline capacity utilization of MVNet integration runtime	MVNetIRPipelineCapacityUtilization	Percent	Maximum	IntegrationRuntimeName	PT1M	Yes
Pipeline waiting queue length of MVNet integration runtime Pipeline waiting queue length of MVNet integration runtime	MVNetIRPipelineWaitingQueueLength	Count	Average	IntegrationRuntimeName	PT1M	Yes
Cancelled pipeline runs metrics Cancelled pipeline runs metrics	PipelineCancelledRuns	Count	Total	FailureType, CancelledBy, Name	PT1M	Yes
Elapsed Time Pipeline Runs Metrics Elapsed Time Pipeline Runs Metrics	PipelineElapsedTimeRuns	Count	Total	RunId, Name	PT1M	Yes
Failed pipeline runs metrics Failed pipeline runs metrics	PipelineFailedRuns	Count	Total	FailureType, Name	PT1M	Yes
Succeeded pipeline runs metrics Succeeded pipeline runs metrics	PipelineSucceededRuns	Count	Total	FailureType, Name	PT1M	Yes
Total entities count Total entities count	ResourceCount	Count	Maximum	<none>	PT1M	Yes
Cancelled SSIS integration runtime start metrics Cancelled SSIS integration runtime start metrics	SSISIntegrationRuntimeStartCancel	Count	Total	IntegrationRuntimeName	PT1M	Yes
Failed SSIS integration runtime start metrics Failed SSIS integration runtime start metrics	SSISIntegrationRuntimeStartFailed	Count	Total	IntegrationRuntimeName	PT1M	Yes
Succeeded SSIS integration runtime start metrics Succeeded SSIS integration runtime start metrics	SSISIntegrationRuntimeStartSucceeded	Count	Total	IntegrationRuntimeName	PT1M	Yes
Stuck SSIS integration runtime stop metrics Stuck SSIS integration runtime stop metrics	SSISIntegrationRuntimeStopStuck	Count	Total	IntegrationRuntimeName	PT1M	Yes
Succeeded SSIS integration runtime stop metrics Succeeded SSIS integration runtime stop metrics	SSISIntegrationRuntimeStopSucceeded	Count	Total	IntegrationRuntimeName	PT1M	Yes
Cancelled SSIS package execution metrics Cancelled SSIS package execution metrics	SSISPackageExecutionCancel	Count	Total	IntegrationRuntimeName	PT1M	Yes
Failed SSIS package execution metrics Failed SSIS package execution metrics	SSISPackageExecutionFailed	Count	Total	IntegrationRuntimeName	PT1M	Yes
Succeeded SSIS package execution metrics Succeeded SSIS package execution metrics	SSISPackageExecutionSucceeded	Count	Total	IntegrationRuntimeName	PT1M	Yes
Cancelled trigger runs metrics Cancelled trigger runs metrics	TriggerCancelledRuns	Count	Total	Name, FailureType	PT1M	Yes
Failed trigger runs metrics Failed trigger runs metrics	TriggerFailedRuns	Count	Total	Name, FailureType	PT1M	Yes
Succeeded trigger runs metrics Succeeded trigger runs metrics	TriggerSucceededRuns	Count	Total	Name, FailureType	PT1M	Yes

Note

Except for PipelineElapsedTimeRuns, only events from completed, triggered activity and pipeline runs are emitted. In-progress and debug runs aren't emitted. However, events from all SSIS package executions are emitted, including those that are completed and in progress, regardless of their invocation methods. For example, you can invoke package executions on Azure-enabled SQL Server Data Tools, via T-SQL on SQL Server Management Studio, SQL Server Agent, or other designated tools, and as triggered or debug runs of Execute SSIS Package activities in Data Factory pipelines.

Metric dimensions

For information about what metric dimensions are, see Multi-dimensional metrics.

This service has the following dimensions associated with its metrics.

Microsoft.DataFactory/datafactories

• pipelineName
• activityName

Microsoft.DataFactory/factories

• ActivityType
• PipelineName
• FailureType
• Name
• IntegrationRuntimeName
• ContainerName
• DagFile
• DagId
• ComputeNodeSize
• Job
• Operator
• Pool
• TaskId
• State
• NodeName
• CancelledBy
• RunId

Resource logs

This section lists the types of resource logs you can collect for this service. The section pulls from the list of all resource logs category types supported in Azure Monitor.

Supported resource logs for Microsoft.DataFactory/factories

Category	Category display name	Log table	Supports basic log plan	Supports ingestion-time transformation	Example queries	Costs to export
ActivityRuns	Pipeline activity runs log	ADFActivityRun	No	Yes	Queries	No
AirflowDagProcessingLogs	Airflow dag processing logs	AirflowDagProcessingLogs ADF Airflow dag processing logs	No	Yes		Yes
AirflowSchedulerLogs	Airflow scheduler logs	ADFAirflowSchedulerLogs ADF Airflow scheduler logs	No	Yes		Yes
AirflowTaskLogs	Airflow task execution logs	ADFAirflowTaskLogs ADF Airflow task logs	No	Yes		Yes
AirflowWebLogs	Airflow web logs	ADFAirflowWebLogs ADF Airflow web logs	No	Yes		Yes
AirflowWorkerLogs	Airflow worker logs	ADFAirflowWorkerLogs ADF Airflow worker logs	No	Yes		Yes
PipelineRuns	Pipeline runs log	ADFPipelineRun	No	Yes	Queries	No
SandboxActivityRuns	Sandbox Activity runs log	ADFSandboxActivityRun	No	Yes		Yes
SandboxPipelineRuns	Sandbox Pipeline runs log	ADFSandboxPipelineRun	No	Yes		Yes
SSISIntegrationRuntimeLogs	SSIS integration runtime logs	ADFSSISIntegrationRuntimeLogs ADF SSIS integration runtime logs	No	Yes		No
SSISPackageEventMessageContext	SSIS package event message context	ADFSSISPackageEventMessageContext ADF SSIS package execution event message context	No	Yes		No
SSISPackageEventMessages	SSIS package event messages	ADFSSISPackageEventMessages ADF SSIS package execution event messages	No	Yes		No
SSISPackageExecutableStatistics	SSIS package executable statistics	ADFSSISPackageExecutableStatistics ADF SSIS package execution executable statistics	No	Yes		No
SSISPackageExecutionComponentPhases	SSIS package execution component phases	ADFSSISPackageExecutionComponentPhases ADF SSIS package execution component phases	No	Yes		No
SSISPackageExecutionDataStatistics	SSIS package exeution data statistics	ADFSSISPackageExecutionDataStatistics ADF SSIS package execution data statistics	No	Yes		No
TriggerRuns	Trigger runs log	ADFTriggerRun	No	Yes	Queries	No

Azure Monitor Logs tables

This section lists the Azure Monitor Logs tables relevant to this service, which are available for query by Log Analytics using Kusto queries. The tables contain resource log data and possibly more depending on what is collected and routed to them.

Data factories

Microsoft.DataFactory/factories

• AzureActivity
• AzureMetrics
• AzureDiagnostics
• ADFActivityRun
• ADFPipelineRun
• ADFTriggerRun
• ADFSandboxActivityRun
• ADFSandboxPipelineRun
• ADFSSISIntegrationRuntimeLogs
• ADFSSISPackageEventMessageContext
• ADFSSISPackageEventMessages
• ADFSSISPackageExecutableStatistics
• ADFSSISPackageExecutionComponentPhases
• ADFSSISPackageExecutionDataStatistics

Monitor and Log Analytics schemas for logs and events

This section lists the attributes that the Azure Monitor and Log Analytics schemas use for monitoring logs and events.

Log Analytics inherits the schema from Azure Monitor with the following exceptions:

• The first letter in each column name is capitalized. For example, the column name correlationId in Monitor is CorrelationId in Log Analytics.

• There's no Level column.

• The dynamic properties column is preserved as the following dynamic JSON blob type.

  Azure Monitor column	Log Analytics column	Type
  $.properties.UserProperties	UserProperties	Dynamic
  $.properties.Annotations	Annotations	Dynamic
  $.properties.Input	Input	Dynamic
  $.properties.Output	Output	Dynamic
  $.properties.Error.errorCode	ErrorCode	int
  $.properties.Error.message	ErrorMessage	string
  $.properties.Error	Error	Dynamic
  $.properties.Predecessors	Predecessors	Dynamic
  $.properties.Parameters	Parameters	Dynamic
  $.properties.SystemParameters	SystemParameters	Dynamic
  $.properties.Tags	Tags	Dynamic

Activity-run log attributes

{
   "Level": "",
   "correlationId":"",
   "time":"",
   "activityRunId":"",
   "pipelineRunId":"",
   "resourceId":"",
   "category":"ActivityRuns",
   "level":"Informational",
   "operationName":"",
   "pipelineName":"",
   "activityName":"",
   "start":"",
   "end":"",
   "properties":
       {
          "Input": "{
              "source": {
                "type": "BlobSource"
              },
              "sink": {
                "type": "BlobSink"
              }
           }",
          "Output": "{"dataRead":121,"dataWritten":121,"copyDuration":5,
               "throughput":0.0236328132,"errors":[]}",
          "Error": "{
              "errorCode": "null",
              "message": "null",
              "failureType": "null",
              "target": "CopyBlobtoBlob"
        }
    }
}

Property	Type	Description	Example
Level	String	The level of the diagnostic logs. For activity-run logs, set the property value to 4.	4
correlationId	String	The unique ID for tracking a particular request.	319dc6b4-f348-405e-b8d7-aafc77b73e77
time	String	The time of the event in the timespan UTC format YYYY-MM-DDTHH:MM:SS.00000Z.	2017-06-28T21:00:27.3534352Z
activityRunId	String	The ID of the activity run.	3a171e1f-b36e-4b80-8a54-5625394f4354
pipelineRunId	String	The ID of the pipeline run.	9f6069d6-e522-4608-9f99-21807bfc3c70
resourceId	String	The ID associated with the data factory resource.	/SUBSCRIPTIONS/<subID>/RESOURCEGROUPS/<resourceGroupName>/PROVIDERS/MICROSOFT.DATAFACTORY/FACTORIES/<dataFactoryName>
category	String	The category of the diagnostic logs. Set the property value to ActivityRuns.	ActivityRuns
level	String	The level of the diagnostic logs. Set the property value to Informational.	Informational
operationName	String	The name of the activity with its status. If the activity is the start heartbeat, the property value is MyActivity -. If the activity is the end heartbeat, the property value is MyActivity - Succeeded.	MyActivity - Succeeded
pipelineName	String	The name of the pipeline.	MyPipeline
activityName	String	The name of the activity.	MyActivity
start	String	The start time of the activity runs in timespan UTC format.	2017-06-26T20:55:29.5007959Z
end	String	The end time of the activity runs in timespan UTC format. If the diagnostic log shows that an activity has started but not yet ended, the property value is 1601-01-01T00:00:00Z.	2017-06-26T20:55:29.5007959Z

Pipeline-run log attributes

{
   "Level": "",
   "correlationId":"",
   "time":"",
   "runId":"",
   "resourceId":"",
   "category":"PipelineRuns",
   "level":"Informational",
   "operationName":"",
   "pipelineName":"",
   "start":"",
   "end":"",
   "status":"",
   "location": "",
   "properties":
    {
      "Parameters": {
        "<parameter1Name>": "<parameter1Value>"
      },
      "SystemParameters": {
        "ExecutionStart": "",
        "TriggerId": "",
        "SubscriptionId": ""
      },
      "Predecessors": [
            {
                "Name": "",
                "Id": "",
                "InvokedByType": ""
            }
        ]
    }
}

Property	Type	Description	Example
Level	String	The level of the diagnostic logs. For activity-run logs, set the property value to 4.	4
correlationId	String	The unique ID for tracking a particular request.	319dc6b4-f348-405e-b8d7-aafc77b73e77
time	String	The time of the event in the timespan UTC format YYYY-MM-DDTHH:MM:SS.00000Z.	2017-06-28T21:00:27.3534352Z
runId	String	The ID of the pipeline run.	9f6069d6-e522-4608-9f99-21807bfc3c70
resourceId	String	The ID associated with the data factory resource.	/SUBSCRIPTIONS/<subID>/RESOURCEGROUPS/<resourceGroupName>/PROVIDERS/MICROSOFT.DATAFACTORY/FACTORIES/<dataFactoryName>
category	String	The category of the diagnostic logs. Set the property value to PipelineRuns.	PipelineRuns
level	String	The level of the diagnostic logs. Set the property value to Informational.	Informational
operationName	String	The name of the pipeline along with its status. After the pipeline run is finished, the property value is Pipeline - Succeeded.	MyPipeline - Succeeded.
pipelineName	String	The name of the pipeline.	MyPipeline
start	String	The start time of the activity runs in timespan UTC format.	2017-06-26T20:55:29.5007959Z.
end	String	The end time of the activity runs in timespan UTC format. If the diagnostic log shows an activity has started but not yet ended, the property value is 1601-01-01T00:00:00Z.	2017-06-26T20:55:29.5007959Z
status	String	The final status of the pipeline run. Possible property values are Succeeded and Failed.	Succeeded
location	String	The Azure region of the pipeline run.	eastasia
predecessors	String	The calling object of the pipeline along with ID.	Manual

Trigger-run log attributes

{
   "Level": "",
   "correlationId":"",
   "time":"",
   "triggerId":"",
   "resourceId":"",
   "category":"TriggerRuns",
   "level":"Informational",
   "operationName":"",
   "triggerName":"",
   "triggerType":"",
   "triggerEvent":"",
   "start":"",
   "status":"",
   "properties":
   {
      "Parameters": {
        "TriggerTime": "",
       "ScheduleTime": ""
      },
      "SystemParameters": {}
    }
}

Property	Type	Description	Example
Level	String	The level of the diagnostic logs. For activity-run logs, set the property value to 4.	4
correlationId	String	The unique ID for tracking a particular request.	319dc6b4-f348-405e-b8d7-aafc77b73e77
time	String	The time of the event in the timespan UTC format YYYY-MM-DDTHH:MM:SS.00000Z.	2017-06-28T21:00:27.3534352Z
triggerId	String	The ID of the trigger run.	08587023010602533858661257311
resourceId	String	The ID associated with the data factory resource.	/SUBSCRIPTIONS/<subID>/RESOURCEGROUPS/<resourceGroupName>/PROVIDERS/MICROSOFT.DATAFACTORY/FACTORIES/<dataFactoryName>
category	String	The category of the diagnostic logs. Set the property value to PipelineRuns.	PipelineRuns
level	String	The level of the diagnostic logs. Set the property value to Informational.	Informational
operationName	String	The name of the trigger with its final status, which indicates whether the trigger successfully fired. If the heartbeat was successful, the property value is MyTrigger - Succeeded.	MyTrigger - Succeeded
triggerName	String	The name of the trigger.	MyTrigger
triggerType	String	The type of the trigger. Possible property values are Manual Trigger and Schedule Trigger.	ScheduleTrigger
triggerEvent	String	The event of the trigger.	ScheduleTime - 2017-07-06T01:50:25Z
start	String	The start time of the trigger firing in timespan UTC format.	2017-06-26T20:55:29.5007959Z
status	String	The final status showing whether the trigger successfully fired. Possible property values are Succeeded and Failed.	Succeeded

SSIS integration runtime log attributes

Here are the log attributes of SQL Server Integration Services (SSIS) integration runtime (IR) start, stop, and maintenance operations.

{
   "time": "",
   "operationName": "",
   "category": "",
   "correlationId": "",
   "dataFactoryName": "",
   "integrationRuntimeName": "",
   "level": "",
   "resultType": "",
   "properties": {
      "message": ""
   },
   "resourceId": ""
}

Property	Type	Description	Example
time	String	The time of event in UTC format: YYYY-MM-DDTHH:MM:SS.00000Z	2017-06-28T21:00:27.3534352Z
operationName	String	The name of your SSIS IR operation	Start/Stop/Maintenance/Heartbeat
category	String	The category of diagnostic logs	SSISIntegrationRuntimeLogs
correlationId	String	The unique ID for tracking a particular operation	f13b159b-515f-4885-9dfa-a664e949f785Deprovision0059035558
dataFactoryName	String	The name of your data factory	MyADFv2
integrationRuntimeName	String	The name of your SSIS IR	MySSISIR
level	String	The level of diagnostic logs	Informational
resultType	String	The result of your SSIS IR operation	Started/InProgress/Succeeded/Failed/Healthy/Unhealthy
message	String	The output message of your SSIS IR operation	The stopping of your SSIS integration runtime has succeeded.
resourceId	String	The unique ID of your data factory resource	/SUBSCRIPTIONS/<subscriptionID>/RESOURCEGROUPS/<resourceGroupName>/PROVIDERS/MICROSOFT.DATAFACTORY/FACTORIES/<dataFactoryName>

SSIS event message context log attributes

Here are the log attributes of conditions related to event messages that are generated by SSIS package executions on your SSIS IR. They convey similar information as an SSIS catalog (SSISDB) event message context table or view that shows runtime values of many SSIS package properties. They're generated when you select Basic/Verbose logging level and useful for debugging or compliance checking.

{
   "time": "",
   "operationName": "",
   "category": "",
   "correlationId": "",
   "dataFactoryName": "",
   "integrationRuntimeName": "",
   "level": "",
   "properties": {
      "operationId": "",
      "contextDepth": "",
      "packagePath": "",
      "contextType": "",
      "contextSourceName": "",
      "contextSourceId": "",
      "propertyName": "",
      "propertyValue": ""
   },
   "resourceId": ""
}

Property	Type	Description	Example
time	String	The time of event in UTC format: YYYY-MM-DDTHH:MM:SS.00000Z	2017-06-28T21:00:27.3534352Z
operationName	String	Set to YourSSISIRName-SSISPackageEventMessageContext	mysqlmissisir-SSISPackageEventMessageContext
category	String	The category of diagnostic logs	SSISPackageEventMessageContext
correlationId	String	The unique ID for tracking a particular operation	e55700df-4caf-4e7c-bfb8-78ac7d2f28a0
dataFactoryName	String	The name of your data factory	MyADFv2
integrationRuntimeName	String	The name of your SSIS IR	MySSISIR
level	String	The level of diagnostic logs	Informational
operationId	String	The unique ID for tracking a particular operation in SSISDB	1 (1 signifies operations related to packages not stored in SSISDB/invoked via T-SQL.)
contextDepth	String	The depth of your event message context	0 (0 signifies the context before package execution starts, 1 signifies the context when an error occurs, and it increases as the context is further from the error.)
packagePath	String	The path of package object as your event message context source	\Package
contextType	String	The type of package object as your event message context source	60 (See more context types.)
contextSourceName	String	The name of package object as your event message context source	MyPackage
contextSourceId	String	The unique ID of package object as your event message context source	{E2CF27FB-EA48-41E9-AF6F-3FE938B4ADE1}
propertyName	String	The name of package property for your event message context source	DelayValidation
propertyValue	String	The value of package property for your event message context source	False
resourceId	String	The unique ID of your data factory resource	/SUBSCRIPTIONS/<subscriptionID>/RESOURCEGROUPS/<resourceGroupName>/PROVIDERS/MICROSOFT.DATAFACTORY/FACTORIES/<dataFactoryName>

SSIS event messages log attributes

Here are the log attributes of event messages that are generated by SSIS package executions on your SSIS IR. They convey similar information as an SSISDB event messages table or view that shows the detailed text or metadata of event messages. They're generated at any logging level except None.

{
   "time": "",
   "operationName": "",
   "category": "",
   "correlationId": "",
   "dataFactoryName": "",
   "integrationRuntimeName": "",
   "level": "",
   "properties": {
      "operationId": "",
      "messageTime": "",
      "messageType": "",
      "messageSourceType": "",
      "message": "",
      "packageName": "",
      "eventName": "",
      "messageSourceName": "",
      "messageSourceId": "",
      "subcomponentName": "",
      "packagePath": "",
      "executionPath": "",
      "threadId": ""
   }
}

Property	Type	Description	Example
time	String	The time of event in UTC format: YYYY-MM-DDTHH:MM:SS.00000Z	2017-06-28T21:00:27.3534352Z
operationName	String	Set to YourSSISIRName-SSISPackageEventMessages	mysqlmissisir-SSISPackageEventMessages
category	String	The category of diagnostic logs	SSISPackageEventMessages
correlationId	String	The unique ID for tracking a particular operation	e55700df-4caf-4e7c-bfb8-78ac7d2f28a0
dataFactoryName	String	The name of your data factory	MyADFv2
integrationRuntimeName	String	The name of your SSIS IR	MySSISIR
level	String	The level of diagnostic logs	Informational
operationId	String	The unique ID for tracking a particular operation in SSISDB	1 (1 signifies operations related to packages not stored in SSISDB/invoked via T-SQL.)
messageTime	String	The time when your event message is created in UTC format	2017-06-28T21:00:27.3534352Z
messageType	String	The type of your event message	70 (See more message types.)
messageSourceType	String	The type of your event message source	20 (See more message source types.)
message	String	The text of your event message	MyPackage:Validation has started.
packageName	String	The name of your executed package file	MyPackage.dtsx
eventName	String	The name of related run-time event	OnPreValidate
messageSourceName	String	The name of package component as your event message source	Data Flow Task
messageSourceId	String	The unique ID of package component as your event message source	{1a45a5a4-3df9-4f02-b818-ebf583829ad2}
subcomponentName	String	The name of data flow component as your event message source	SSIS.Pipeline
packagePath	String	The path of package object as your event message source	\Package\Data Flow Task
executionPath	String	The full path from parent package to executed component	\Transformation\Data Flow Task (This path also captures component iterations.)
threadId	String	The unique ID of thread executed when your event message is logged	{1a45a5a4-3df9-4f02-b818-ebf583829ad2}

SSIS executable statistics log attributes

Here are the log attributes of executable statistics that are generated by SSIS package executions on your SSIS IR, where executables are containers or tasks in the control flow of packages. They convey similar information as an SSISDB executable statistics table or view that shows a row for each running executable, including its iterations. They're generated at any logging level except None and useful for identifying task-level bottlenecks or failures.

{
   "time": "",
   "operationName": "",
   "category": "",
   "correlationId": "",
   "dataFactoryName": "",
   "integrationRuntimeName": "",
   "level": "",
   "properties": {
      "executionId": "",
      "executionPath": "",
      "startTime": "",
      "endTime": "",
      "executionDuration": "",
      "executionResult": "",
      "executionValue": ""
   },
   "resourceId": ""
}

Property	Type	Description	Example
time	String	The time of event in UTC format: YYYY-MM-DDTHH:MM:SS.00000Z	2017-06-28T21:00:27.3534352Z
operationName	String	Set to YourSSISIRName-SSISPackageExecutableStatistics	mysqlmissisir-SSISPackageExecutableStatistics
category	String	The category of diagnostic logs	SSISPackageExecutableStatistics
correlationId	String	The unique ID for tracking a particular operation	e55700df-4caf-4e7c-bfb8-78ac7d2f28a0
dataFactoryName	String	The name of your data factory	MyADFv2
integrationRuntimeName	String	The name of your SSIS IR	MySSISIR
level	String	The level of diagnostic logs	Informational
executionId	String	The unique ID for tracking a particular execution in SSISDB	1 (1 signifies executions related to packages not stored in SSISDB/invoked via T-SQL.)
executionPath	String	The full path from parent package to executed component	\Transformation\Data Flow Task (This path also captures component iterations.)
startTime	String	The time when executable enters pre-execute phase in UTC format	2017-06-28T21:00:27.3534352Z
endTime	String	The time when executable enters post-execute phase in UTC format	2017-06-28T21:00:27.3534352Z
executionDuration	String	The running time of executable in milliseconds	1,125
executionResult	String	The result of running executable	0 (0 signifies success, 1 signifies failure, 2 signifies completion, and 3 signifies cancellation.)
executionValue	String	The user-defined value returned by running executable	1
resourceId	String	The unique ID of your data factory resource	/SUBSCRIPTIONS/<subscriptionID>/RESOURCEGROUPS/<resourceGroupName>/PROVIDERS/MICROSOFT.DATAFACTORY/FACTORIES/<dataFactoryName>

SSIS execution component phases log attributes

Here are the log attributes of runtime statistics for data flow components that are generated by SSIS package executions on your SSIS IR. They convey similar information as an SSISDB execution component phases table or view that shows the time spent by data flow components in all their execution phases. They're generated when you select Performance/Verbose logging level and useful for capturing data flow execution statistics.

{
   "time": "",
   "operationName": "",
   "category": "",
   "correlationId": "",
   "dataFactoryName": "",
   "integrationRuntimeName": "",
   "level": "",
   "properties": {
      "executionId": "",
      "packageName": "",
      "taskName": "",
      "subcomponentName": "",
      "phase": "",
      "startTime": "",
      "endTime": "",
      "executionPath": ""
   },
   "resourceId": ""
}

Property	Type	Description	Example
time	String	The time of event in UTC format: YYYY-MM-DDTHH:MM:SS.00000Z	2017-06-28T21:00:27.3534352Z
operationName	String	Set to YourSSISIRName-SSISPackageExecutionComponentPhases	mysqlmissisir-SSISPackageExecutionComponentPhases
category	String	The category of diagnostic logs	SSISPackageExecutionComponentPhases
correlationId	String	The unique ID for tracking a particular operation	e55700df-4caf-4e7c-bfb8-78ac7d2f28a0
dataFactoryName	String	The name of your data factory	MyADFv2
integrationRuntimeName	String	The name of your SSIS IR	MySSISIR
level	String	The level of diagnostic logs	Informational
executionId	String	The unique ID for tracking a particular execution in SSISDB	1 (1 signifies executions related to packages not stored in SSISDB/invoked via T-SQL.)
packageName	String	The name of your executed package file	MyPackage.dtsx
taskName	String	The name of executed data flow task	Data Flow Task
subcomponentName	String	The name of data flow component	Derived Column
phase	String	The name of execution phase	AcquireConnections
startTime	String	The time when execution phase starts in UTC format	2017-06-28T21:00:27.3534352Z
endTime	String	The time when execution phase ends in UTC format	2017-06-28T21:00:27.3534352Z
executionPath	String	The path of execution for data flow task	\Transformation\Data Flow Task
resourceId	String	The unique ID of your data factory resource	/SUBSCRIPTIONS/<subscriptionID>/RESOURCEGROUPS/<resourceGroupName>/PROVIDERS/MICROSOFT.DATAFACTORY/FACTORIES/<dataFactoryName>

SSIS execution data statistics log attributes

Here are the log attributes of data movements through each leg of data flow pipelines, from upstream to downstream components, that are generated by SSIS package executions on your SSIS IR. They convey similar information as an SSISDB execution data statistics table or view that shows row counts of data moved through data flow tasks. They're generated when you select Verbose logging level and useful for computing data flow throughput.

{
   "time": "",
   "operationName": "",
   "category": "",
   "correlationId": "",
   "dataFactoryName": "",
   "integrationRuntimeName": "",
   "level": "",
   "properties": {
      "executionId": "",
      "packageName": "",
      "taskName": "",
      "dataflowPathIdString": "",
      "dataflowPathName": "",
      "sourceComponentName": "",
      "destinationComponentName": "",
      "rowsSent": "",
      "createdTime": "",
      "executionPath": ""
   },
   "resourceId": ""
}

Property	Type	Description	Example
time	String	The time of event in UTC format: YYYY-MM-DDTHH:MM:SS.00000Z	2017-06-28T21:00:27.3534352Z
operationName	String	Set to YourSSISIRName-SSISPackageExecutionDataStatistics	mysqlmissisir-SSISPackageExecutionDataStatistics
category	String	The category of diagnostic logs	SSISPackageExecutionDataStatistics
correlationId	String	The unique ID for tracking a particular operation	e55700df-4caf-4e7c-bfb8-78ac7d2f28a0
dataFactoryName	String	The name of your data factory	MyADFv2
integrationRuntimeName	String	The name of your SSIS IR	MySSISIR
level	String	The level of diagnostic logs	Informational
executionId	String	The unique ID for tracking a particular execution in SSISDB	1 (1 signifies executions related to packages not stored in SSISDB/invoked via T-SQL.)
packageName	String	The name of your executed package file	MyPackage.dtsx
taskName	String	The name of executed data flow task	Data Flow Task
dataflowPathIdString	String	The unique ID for tracking data flow path	Paths[SQLDB Table3.ADO NET Source Output]
dataflowPathName	String	The name of data flow path	ADO NET Source Output
sourceComponentName	String	The name of data flow component that sends data	SQLDB Table3
destinationComponentName	String	The name of data flow component that receives data	Derived Column
rowsSent	String	The number of rows sent by source component	500
createdTime	String	The time when row values are obtained in UTC format	2017-06-28T21:00:27.3534352Z
executionPath	String	The path of execution for data flow task	\Transformation\Data Flow Task
resourceId	String	The unique ID of your data factory resource	/SUBSCRIPTIONS/<subscriptionID>/RESOURCEGROUPS/<resourceGroupName>/PROVIDERS/MICROSOFT.DATAFACTORY/FACTORIES/<dataFactoryName>

Activity log

The linked table lists the operations that can be recorded in the activity log for this service. These operations are a subset of all the possible resource provider operations in the activity log.

For more information on the schema of activity log entries, see Activity Log schema.

• Microsoft.DataFactory resource provider operations

Related content

• Monitor Data Factory
• Monitor Azure resources with Azure Monitor
• Monitor programmatically using SDKs