Azure Machine Learning monitoring data reference

This article contains all the monitoring reference information for this service.

See Monitor Machine Learning for details on the data you can collect for Azure Machine Learning and how to use it.

Metrics

This section lists all the automatically collected platform metrics for this service. These metrics are also part of the global list of all platform metrics supported in Azure Monitor.

For information on metric retention, see Azure Monitor Metrics overview.

The resource provider for these metrics is Microsoft.MachineLearningServices/workspaces.

The metrics categories are Model, Quota, Resource, Run, and Traffic. Quota information is for Machine Learning compute only. Run provides information on training runs for the workspace.

Supported metrics for Microsoft.MachineLearningServices/workspaces

The following table lists the metrics available for the Microsoft.MachineLearningServices/workspaces resource type.

• All columns might not be present in every table.
• Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.

Table headings

• Category - The metrics group or classification.
• Metric - The metric display name as it appears in the Azure portal.
• Name in REST API - The metric name as referred to in the REST API.
• Unit - Unit of measure.
• Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
• Dimensions - Dimensions available for the metric.
• Time Grains - Intervals at which the metric is sampled. For example, PT1M indicates that the metric is sampled every minute, PT30M every 30 minutes, PT1H every hour, and so on.
• DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.

Category: Agents

Metric	Name in REST API	Advanced platform metrics	Unit	Aggregation	Dimensions	Time Grains	DS Export
Agents Number of events for AI Agents in this workspace	Agents	No	Count	Average, Maximum, Minimum, Total (Sum)	EventType	PT1M	No
IndexedFiles Number of files indexed for file search in this workspace	IndexedFiles	No	Count	Average, Maximum, Minimum, Total (Sum)	ErrorCode, Status, VectorStoreId	PT1M	No
Messages Number of events for AI Agent messages in this workspace	Messages	No	Count	Average, Maximum, Minimum, Total (Sum)	EventType, ThreadId	PT1M	No
Runs Number of runs by AI Agents in this workspace	Runs	No	Count	Average, Maximum, Minimum, Total (Sum)	AgentId, RunStatus, StatusCode, StreamType	PT1M	No
Threads Number of events for AI Agent threads in this workspace	Threads	No	Count	Average, Maximum, Minimum, Total (Sum)	EventType	PT1M	No
Tokens Count of tokens by AI Agents in this workspace	Tokens	No	Count	Average, Maximum, Minimum, Total (Sum)	AgentId, TokenType	PT1M	No
ToolCalls Tool calls made by AI Agents in this workspace	ToolCalls	No	Count	Average, Maximum, Minimum, Total (Sum)	AgentId, ToolName	PT1M	No

Category: Model

Metric	Name in REST API	Advanced platform metrics	Unit	Aggregation	Dimensions	Time Grains	DS Export
Model Deploy Failed Number of model deployments that failed in this workspace	Model Deploy Failed	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, StatusCode	PT1M	Yes
Model Deploy Started Number of model deployments started in this workspace	Model Deploy Started	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario	PT1M	Yes
Model Deploy Succeeded Number of model deployments that succeeded in this workspace	Model Deploy Succeeded	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario	PT1M	Yes
Model Register Failed Number of model registrations that failed in this workspace	Model Register Failed	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, StatusCode	PT1M	Yes
Model Register Succeeded Number of model registrations that succeeded in this workspace	Model Register Succeeded	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario	PT1M	Yes

Category: Quota

Metric	Name in REST API	Advanced platform metrics	Unit	Aggregation	Dimensions	Time Grains	DS Export
Active Cores Number of active cores	Active Cores	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Active Nodes Number of Acitve nodes. These are the nodes which are actively running a job.	Active Nodes	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Idle Cores Number of idle cores	Idle Cores	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Idle Nodes Number of idle nodes. Idle nodes are the nodes which are not running any jobs but can accept new job if available.	Idle Nodes	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Leaving Cores Number of leaving cores	Leaving Cores	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Leaving Nodes Number of leaving nodes. Leaving nodes are the nodes which just finished processing a job and will go to Idle state.	Leaving Nodes	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Preempted Cores Number of preempted cores	Preempted Cores	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Preempted Nodes Number of preempted nodes. These nodes are the low priority nodes which are taken away from the available node pool.	Preempted Nodes	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Quota Utilization Percentage Percent of quota utilized	Quota Utilization Percentage	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName, VmFamilyName, VmPriority	PT1M	Yes
Total Cores Number of total cores	Total Cores	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Total Nodes Number of total nodes. This total includes some of Active Nodes, Idle Nodes, Unusable Nodes, Premepted Nodes, Leaving Nodes	Total Nodes	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Unusable Cores Number of unusable cores	Unusable Cores	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes
Unusable Nodes Number of unusable nodes. Unusable nodes are not functional due to some unresolvable issue. Azure will recycle these nodes.	Unusable Nodes	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, ClusterName	PT1M	Yes

Category: Resource

Metric	Name in REST API	Advanced platform metrics	Unit	Aggregation	Dimensions	Time Grains	DS Export
CpuCapacityMillicores Maximum capacity of a CPU node in millicores. Capacity is aggregated in one minute intervals.	CpuCapacityMillicores	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
CpuMemoryCapacityMegabytes Maximum memory utilization of a CPU node in megabytes. Utilization is aggregated in one minute intervals.	CpuMemoryCapacityMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
CpuMemoryUtilizationMegabytes Memory utilization of a CPU node in megabytes. Utilization is aggregated in one minute intervals.	CpuMemoryUtilizationMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
CpuMemoryUtilizationPercentage Memory utilization percentage of a CPU node. Utilization is aggregated in one minute intervals.	CpuMemoryUtilizationPercentage	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
CpuUtilization Percentage of utilization on a CPU node. Utilization is reported at one minute intervals.	CpuUtilization	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, runId, NodeId, ClusterName	PT1M	Yes
CpuUtilizationMillicores Utilization of a CPU node in millicores. Utilization is aggregated in one minute intervals.	CpuUtilizationMillicores	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
CpuUtilizationPercentage Utilization percentage of a CPU node. Utilization is aggregated in one minute intervals.	CpuUtilizationPercentage	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
DiskAvailMegabytes Available disk space in megabytes. Metrics are aggregated in one minute intervals.	DiskAvailMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
DiskReadMegabytes Data read from disk in megabytes. Metrics are aggregated in one minute intervals.	DiskReadMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
DiskUsedMegabytes Used disk space in megabytes. Metrics are aggregated in one minute intervals.	DiskUsedMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
DiskWriteMegabytes Data written into disk in megabytes. Metrics are aggregated in one minute intervals.	DiskWriteMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
GpuCapacityMilliGPUs Maximum capacity of a GPU device in milli-GPUs. Capacity is aggregated in one minute intervals.	GpuCapacityMilliGPUs	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, DeviceId, ComputeName	PT1M	Yes
GpuEnergyJoules Interval energy in Joules on a GPU node. Energy is reported at one minute intervals.	GpuEnergyJoules	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, runId, rootRunId, InstanceId, DeviceId, ComputeName	PT1M	Yes
GpuMemoryCapacityMegabytes Maximum memory capacity of a GPU device in megabytes. Capacity aggregated in at one minute intervals.	GpuMemoryCapacityMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, DeviceId, ComputeName	PT1M	Yes
GpuMemoryUtilization Percentage of memory utilization on a GPU node. Utilization is reported at one minute intervals.	GpuMemoryUtilization	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, runId, NodeId, DeviceId, ClusterName	PT1M	Yes
GpuMemoryUtilizationMegabytes Memory utilization of a GPU device in megabytes. Utilization aggregated in at one minute intervals.	GpuMemoryUtilizationMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, DeviceId, ComputeName	PT1M	Yes
GpuMemoryUtilizationPercentage Memory utilization percentage of a GPU device. Utilization aggregated in at one minute intervals.	GpuMemoryUtilizationPercentage	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, DeviceId, ComputeName	PT1M	Yes
GpuUtilization Percentage of utilization on a GPU node. Utilization is reported at one minute intervals.	GpuUtilization	No	Count	Average, Maximum, Minimum, Total (Sum)	Scenario, runId, NodeId, DeviceId, ClusterName	PT1M	Yes
GpuUtilizationMilliGPUs Utilization of a GPU device in milli-GPUs. Utilization is aggregated in one minute intervals.	GpuUtilizationMilliGPUs	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, DeviceId, ComputeName	PT1M	Yes
GpuUtilizationPercentage Utilization percentage of a GPU device. Utilization is aggregated in one minute intervals.	GpuUtilizationPercentage	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, DeviceId, ComputeName	PT1M	Yes
IBReceiveMegabytes Network data received over InfiniBand in megabytes. Metrics are aggregated in one minute intervals.	IBReceiveMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName, DeviceId	PT1M	Yes
IBTransmitMegabytes Network data sent over InfiniBand in megabytes. Metrics are aggregated in one minute intervals.	IBTransmitMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName, DeviceId	PT1M	Yes
NetworkInputMegabytes Network data received in megabytes. Metrics are aggregated in one minute intervals.	NetworkInputMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName, DeviceId	PT1M	Yes
NetworkOutputMegabytes Network data sent in megabytes. Metrics are aggregated in one minute intervals.	NetworkOutputMegabytes	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName, DeviceId	PT1M	Yes
StorageAPIFailureCount Azure Blob Storage API calls failure count.	StorageAPIFailureCount	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes
StorageAPISuccessCount Azure Blob Storage API calls success count.	StorageAPISuccessCount	No	Count	Average, Maximum, Minimum, Total (Sum)	RunId, InstanceId, ComputeName	PT1M	Yes

Category: Run

Metric	Name in REST API	Advanced platform metrics	Unit	Aggregation	Dimensions	Time Grains	DS Export
Cancel Requested Runs Number of runs where cancel was requested for this workspace. Count is updated when cancellation request has been received for a run.	Cancel Requested Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Cancelled Runs Number of runs cancelled for this workspace. Count is updated when a run is successfully cancelled.	Cancelled Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Completed Runs Number of runs completed successfully for this workspace. Count is updated when a run has completed and output has been collected.	Completed Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Errors Number of run errors in this workspace. Count is updated whenever run encounters an error.	Errors	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario	PT1M	Yes
Failed Runs Number of runs failed for this workspace. Count is updated when a run fails.	Failed Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Finalizing Runs Number of runs entered finalizing state for this workspace. Count is updated when a run has completed but output collection still in progress.	Finalizing Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Not Responding Runs Number of runs not responding for this workspace. Count is updated when a run enters Not Responding state.	Not Responding Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Not Started Runs Number of runs in Not Started state for this workspace. Count is updated when a request is received to create a run but run information has not yet been populated.	Not Started Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Preparing Runs Number of runs that are preparing for this workspace. Count is updated when a run enters Preparing state while the run environment is being prepared.	Preparing Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Provisioning Runs Number of runs that are provisioning for this workspace. Count is updated when a run is waiting on compute target creation or provisioning.	Provisioning Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Queued Runs Number of runs that are queued for this workspace. Count is updated when a run is queued in compute target. Can occure when waiting for required compute nodes to be ready.	Queued Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Started Runs Number of runs running for this workspace. Count is updated when run starts running on required resources.	Started Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Starting Runs Number of runs started for this workspace. Count is updated after request to create run and run info, such as the Run Id, has been populated	Starting Runs	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario, RunType, PublishedPipelineId, ComputeType, PipelineStepType, ExperimentName	PT1M	Yes
Warnings Number of run warnings in this workspace. Count is updated whenever a run encounters a warning.	Warnings	No	Count	Total (Sum), Average, Minimum, Maximum, Count	Scenario	PT1M	Yes

Supported metrics for Microsoft.MachineLearningServices/workspaces/onlineEndpoints

The following table lists the metrics available for the Microsoft.MachineLearningServices/workspaces/onlineEndpoints resource type.

• All columns might not be present in every table.
• Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.

Table headings

• Category - The metrics group or classification.
• Metric - The metric display name as it appears in the Azure portal.
• Name in REST API - The metric name as referred to in the REST API.
• Unit - Unit of measure.
• Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
• Dimensions - Dimensions available for the metric.
• Time Grains - Intervals at which the metric is sampled. For example, PT1M indicates that the metric is sampled every minute, PT30M every 30 minutes, PT1H every hour, and so on.
• DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.

Category: Traffic

Metric	Name in REST API	Advanced platform metrics	Unit	Aggregation	Dimensions	Time Grains	DS Export
Connections Active The total number of concurrent TCP connections active from clients.	ConnectionsActive	No	Count	Average	<none>	PT1M	No
Data Collection Errors Per Minute The number of data collection events dropped per minute.	DataCollectionErrorsPerMinute	No	Count	Minimum, Maximum, Average	deployment, reason, type	PT1M	No
Data Collection Events Per Minute The number of data collection events processed per minute.	DataCollectionEventsPerMinute	No	Count	Minimum, Maximum, Average	deployment, type	PT1M	No
Network Bytes The bytes per second served for the endpoint.	NetworkBytes	No	BytesPerSecond	Average	<none>	PT1M	No
New Connections Per Second The average number of new TCP connections per second established from clients.	NewConnectionsPerSecond	No	CountPerSecond	Average	<none>	PT1M	No
Request Latency The average complete interval of time taken for a request to be responded in milliseconds	RequestLatency	No	Milliseconds	Average	deployment	PT1M	Yes
Request Latency P50 The average P50 request latency aggregated by all request latency values collected over the selected time period	RequestLatency_P50	No	Milliseconds	Average	deployment	PT1M	Yes
Request Latency P90 The average P90 request latency aggregated by all request latency values collected over the selected time period	RequestLatency_P90	No	Milliseconds	Average	deployment	PT1M	Yes
Request Latency P95 The average P95 request latency aggregated by all request latency values collected over the selected time period	RequestLatency_P95	No	Milliseconds	Average	deployment	PT1M	Yes
Request Latency P99 The average P99 request latency aggregated by all request latency values collected over the selected time period	RequestLatency_P99	No	Milliseconds	Average	deployment	PT1M	Yes
Requests Per Minute The number of requests sent to online endpoint within a minute	RequestsPerMinute	No	Count	Average	deployment, statusCode, statusCodeClass, modelStatusCode	PT1M	No

Supported metrics for Microsoft.MachineLearningServices/workspaces/onlineEndpoints/deployments

The following table lists the metrics available for the Microsoft.MachineLearningServices/workspaces/onlineEndpoints/deployments resource type.

• All columns might not be present in every table.
• Some columns might be beyond the viewing area of the page. Select Expand table to view all available columns.

Table headings

• Category - The metrics group or classification.
• Metric - The metric display name as it appears in the Azure portal.
• Name in REST API - The metric name as referred to in the REST API.
• Unit - Unit of measure.
• Aggregation - The default aggregation type. Valid values: Average (Avg), Minimum (Min), Maximum (Max), Total (Sum), Count.
• Dimensions - Dimensions available for the metric.
• Time Grains - Intervals at which the metric is sampled. For example, PT1M indicates that the metric is sampled every minute, PT30M every 30 minutes, PT1H every hour, and so on.
• DS Export- Whether the metric is exportable to Azure Monitor Logs via diagnostic settings. For information on exporting metrics, see Create diagnostic settings in Azure Monitor.

Category: Resource

Metric	Name in REST API	Advanced platform metrics	Unit	Aggregation	Dimensions	Time Grains	DS Export
CPU Memory Utilization Percentage Percentage of memory utilization on an instance. Utilization is reported at one minute intervals.	CpuMemoryUtilizationPercentage	No	Percent	Minimum, Maximum, Average	instanceId	PT1M	Yes
CPU Utilization Percentage Percentage of CPU utilization on an instance. Utilization is reported at one minute intervals.	CpuUtilizationPercentage	No	Percent	Minimum, Maximum, Average	instanceId	PT1M	Yes
Data Collection Errors Per Minute The number of data collection events dropped per minute.	DataCollectionErrorsPerMinute	No	Count	Minimum, Maximum, Average	instanceId, reason, type	PT1M	No
Data Collection Events Per Minute The number of data collection events processed per minute.	DataCollectionEventsPerMinute	No	Count	Minimum, Maximum, Average	instanceId, type	PT1M	No
Deployment Capacity The number of instances in the deployment.	DeploymentCapacity	No	Count	Minimum, Maximum, Average	instanceId, State	PT1M	No
Disk Utilization Percentage of disk utilization on an instance. Utilization is reported at one minute intervals.	DiskUtilization	No	Percent	Minimum, Maximum, Average	instanceId, disk	PT1M	Yes
GPU Energy in Joules Interval energy in Joules on a GPU node. Energy is reported at one minute intervals.	GpuEnergyJoules	No	Count	Minimum, Maximum, Average	instanceId	PT1M	No
GPU Memory Utilization Percentage Percentage of GPU memory utilization on an instance. Utilization is reported at one minute intervals.	GpuMemoryUtilizationPercentage	No	Percent	Minimum, Maximum, Average	instanceId	PT1M	Yes
GPU Utilization Percentage Percentage of GPU utilization on an instance. Utilization is reported at one minute intervals.	GpuUtilizationPercentage	No	Percent	Minimum, Maximum, Average	instanceId	PT1M	Yes

Category: Traffic

Metric	Name in REST API	Advanced platform metrics	Unit	Aggregation	Dimensions	Time Grains	DS Export
Request Latency P50 The average P50 request latency aggregated by all request latency values collected over the selected time period	RequestLatency_P50	No	Milliseconds	Average	<none>	PT1M	Yes
Request Latency P90 The average P90 request latency aggregated by all request latency values collected over the selected time period	RequestLatency_P90	No	Milliseconds	Average	<none>	PT1M	Yes
Request Latency P95 The average P95 request latency aggregated by all request latency values collected over the selected time period	RequestLatency_P95	No	Milliseconds	Average	<none>	PT1M	Yes
Request Latency P99 The average P99 request latency aggregated by all request latency values collected over the selected time period	RequestLatency_P99	No	Milliseconds	Average	<none>	PT1M	Yes
Requests Per Minute The number of requests sent to online deployment within a minute	RequestsPerMinute	No	Count	Average	envoy_response_code	PT1M	No

Metric dimensions

For information about what metric dimensions are, see Multi-dimensional metrics.

This service has the following dimensions associated with its metrics.

Dimension	Description
Cluster Name	The name of the compute cluster resource. Available for all quota metrics.
Vm Family Name	The name of the VM family used by the cluster. Available for quota utilization percentage.
Vm Priority	The priority of the VM. Available for quota utilization percentage.
CreatedTime	Only available for CpuUtilization and GpuUtilization.
DeviceId	ID of the device (GPU). Only available for GpuUtilization.
NodeId	ID of the node created where job is running. Only available for CpuUtilization and GpuUtilization.
RunId	ID of the run/job. Only available for CpuUtilization and GpuUtilization.
ComputeType	The compute type that the run used. Only available for Completed runs, Failed runs, and Started runs.
PipelineStepType	The type of PipelineStep used in the run. Only available for Completed runs, Failed runs, and Started runs.
PublishedPipelineId	The ID of the published pipeline used in the run. Only available for Completed runs, Failed runs, and Started runs.
RunType	The type of run. Only available for Completed runs, Failed runs, and Started runs.

The valid values for the RunType dimension are:

Value	Description
Experiment	Non-pipeline runs.
PipelineRun	A pipeline run, which is the parent of a StepRun.
StepRun	A run for a pipeline step.
ReusedStepRun	A run for a pipeline step that reuses a previous run.

Resource logs

This section lists the types of resource logs you can collect for this service. The section pulls from the list of all resource logs category types supported in Azure Monitor.

Supported resource logs for Microsoft.MachineLearningServices/registries

Category	Costs to export	Log table	Supports basic log plan	Supports ingestion-time transformation	Example queries
Registry Asset Read Event	Yes	AmlRegistryReadEventsLog Azure ML Registry Read events log. It keeps records of Read operations with registries data access (data plane), including user identity, asset name and version for each access event.	No	Yes
Registry Asset Write Event	Yes	AmlRegistryWriteEventsLog Azure ML Registry Write events log. It keeps records of Write operations with registries data access (data plane), including user identity, asset name and version for each access event.	No	Yes	Queries

Supported resource logs for Microsoft.MachineLearningServices/workspaces

Category	Costs to export	Log table	Supports basic log plan	Supports ingestion-time transformation	Example queries
AmlComputeClusterEvent	No	AmlComputeClusterEvent AmlCompute Cluster events	No	Yes	Queries
AmlComputeClusterNodeEvent	Yes		No	No
AmlComputeCpuGpuUtilization	No	AmlComputeCpuGpuUtilization Azure Machine Learning services CPU and GPU utilizaion logs.	No	Yes	Queries
AmlComputeJobEvent	No	AmlComputeJobEvent AmlCompute Job events	No	Yes	Queries
AmlRunStatusChangedEvent	No	AmlRunStatusChangedEvent Azure Machine Learning services run status event logs.	No	Yes
ComputeInstanceEvent	Yes	AmlComputeInstanceEvent Events when ML Compute Instance is accessed (read/write).	No	Yes
DataLabelChangeEvent	Yes	AmlDataLabelEvent Events when data label(s) or its projects is accessed (read, created, or deleted).	No	Yes
DataLabelReadEvent	Yes	AmlDataLabelEvent Events when data label(s) or its projects is accessed (read, created, or deleted).	No	Yes
DataSetChangeEvent	Yes	AmlDataSetEvent Events when a registered or unregistered ML datastore is accessed (read, created, or deleted).	No	Yes	Queries
DataSetReadEvent	Yes	AmlDataSetEvent Events when a registered or unregistered ML datastore is accessed (read, created, or deleted).	No	Yes	Queries
DataStoreChangeEvent	Yes	AmlDataStoreEvent Events when ML datastore is accessed (read, created, or deleted).	No	Yes
DataStoreReadEvent	Yes	AmlDataStoreEvent Events when ML datastore is accessed (read, created, or deleted).	No	Yes
DeploymentEventACI	Yes	AmlDeploymentEvent Events when a model deployment happens on ACI or AKS.	No	Yes
DeploymentEventAKS	Yes	AmlDeploymentEvent Events when a model deployment happens on ACI or AKS.	No	Yes
DeploymentReadEvent	Yes	AmlDeploymentEvent Events when a model deployment happens on ACI or AKS.	No	Yes
EnvironmentChangeEvent	Yes	AmlEnvironmentEvent Events when ML environments are accessed (read, created, or deleted).	No	Yes	Queries
EnvironmentReadEvent	Yes	AmlEnvironmentEvent Events when ML environments are accessed (read, created, or deleted).	No	Yes	Queries
InferencingOperationACI	Yes	AmlInferencingEvent Events for inference or related operation on AKS or ACI compute type.	No	Yes
InferencingOperationAKS	Yes	AmlInferencingEvent Events for inference or related operation on AKS or ACI compute type.	No	Yes
Managed Network Events	Yes		No	No
ModelsActionEvent	Yes	AmlModelsEvent Events when ML model is accessed (read, created, or deleted). Incudes events when packaging of models and assets happen into a ready-to-build packages.	No	Yes	Queries
ModelsChangeEvent	Yes	AmlModelsEvent Events when ML model is accessed (read, created, or deleted). Incudes events when packaging of models and assets happen into a ready-to-build packages.	No	Yes	Queries
ModelsReadEvent	Yes	AmlModelsEvent Events when ML model is accessed (read, created, or deleted). Incudes events when packaging of models and assets happen into a ready-to-build packages.	No	Yes	Queries
PipelineChangeEvent	Yes	AmlPipelineEvent Events when ML pipeline draft or endpoint or module are accessed (read, created, or deleted).	No	Yes
PipelineReadEvent	Yes	AmlPipelineEvent Events when ML pipeline draft or endpoint or module are accessed (read, created, or deleted).	No	Yes
RunEvent	Yes	AmlRunEvent Events when ML experiments are accessed (read, created, or deleted).	No	Yes
RunReadEvent	Yes	AmlRunEvent Events when ML experiments are accessed (read, created, or deleted).	No	Yes

Supported resource logs for Microsoft.MachineLearningServices/workspaces/onlineEndpoints

Category	Costs to export	Log table	Supports basic log plan	Supports ingestion-time transformation	Example queries
AmlOnlineEndpointConsoleLog	Yes	AmlOnlineEndpointConsoleLog Azure ML online endpoints console logs. It provides console logs output from user containers.	No	Yes	Queries
AmlOnlineEndpointEventLog	Yes	AmlOnlineEndpointEventLog Azure ML online endpoints event logs. It provides event logs regarding the inference-server container's life cycle.	No	Yes	Queries
AmlOnlineEndpointTrafficLog	Yes	AmlOnlineEndpointTrafficLog Traffic logs for AzureML (machine learning) online endpoints. The table could be used to check the detailed information of the request to an online endpoint. For example, you could use it to check the request duration, the request failure reason, etc.	No	Yes	Queries

Azure Monitor Logs tables

This section lists the Azure Monitor Logs tables relevant to this service, which are available for query by Log Analytics using Kusto queries. The tables contain resource log data and possibly more depending on what is collected and routed to them.

Machine Learning

Microsoft.MachineLearningServices/workspaces

• AzureActivity
• AMLOnlineEndpointConsoleLog
• AMLOnlineEndpointTrafficLog
• AMLOnlineEndpointEventLog
• AzureMetrics
• AMLComputeClusterEvent
• AMLComputeClusterNodeEvent
• AMLComputeJobEvent
• AMLRunStatusChangedEvent
• AMLComputeCpuGpuUtilization
• AMLComputeInstanceEvent
• AMLDataLabelEvent
• AMLDataSetEvent
• AMLDataStoreEvent
• AMLDeploymentEvent
• AMLEnvironmentEvent
• AMLInferencingEvent
• AMLModelsEvent
• AMLPipelineEvent
• AMLRunEvent

Microsoft.MachineLearningServices/registries

• AzureActivity
• AmlRegistryReadEventsLog
• AmlRegistryWriteEventsLog

Activity log

The linked table lists the operations that can be recorded in the activity log for this service. These operations are a subset of all the possible resource provider operations in the activity log.

For more information on the schema of activity log entries, see Activity Log schema.

The following table lists some operations related to Machine Learning that may be created in the activity log. For a complete listing of Microsoft.MachineLearningServices operations, see Microsoft.MachineLearningServices resource provider operations.

Operation	Description
Creates or updates a Machine Learning workspace	A workspace was created or updated
CheckComputeNameAvailability	Check if a compute name is already in use
Creates or updates the compute resources	A compute resource was created or updated
Deletes the compute resources	A compute resource was deleted
List secrets	On operation listed secrets for a Machine Learning workspace

Log schemas

Azure Machine Learning uses the following schemas.

AmlComputeJobEvent table

Property	Description
TimeGenerated	Time when the log entry was generated
OperationName	Name of the operation associated with the log event
Category	Name of the log event
JobId	ID of the Job submitted
ExperimentId	ID of the Experiment
ExperimentName	Name of the Experiment
CustomerSubscriptionId	SubscriptionId where Experiment and Job as submitted
WorkspaceName	Name of the machine learning workspace
ClusterName	Name of the Cluster
ProvisioningState	State of the Job submission
ResourceGroupName	Name of the resource group
JobName	Name of the Job
ClusterId	ID of the cluster
EventType	Type of the Job event. For example, JobSubmitted, JobRunning, JobFailed, JobSucceeded.
ExecutionState	State of the job (the Run). For example, Queued, Running, Succeeded, Failed
ErrorDetails	Details of job error
CreationApiVersion	Api version used to create the job
ClusterResourceGroupName	Resource group name of the cluster
TFWorkerCount	Count of TF workers
TFParameterServerCount	Count of TF parameter server
ToolType	Type of tool used
RunInContainer	Flag describing if job should be run inside a container
JobErrorMessage	detailed message of Job error
NodeId	ID of the node created where job is running

AmlComputeClusterEvent table

Property	Description
TimeGenerated	Time when the log entry was generated
OperationName	Name of the operation associated with the log event
Category	Name of the log event
ProvisioningState	Provisioning state of the cluster
ClusterName	Name of the cluster
ClusterType	Type of the cluster
CreatedBy	User who created the cluster
CoreCount	Count of the cores in the cluster
VmSize	Vm size of the cluster
VmPriority	Priority of the nodes created inside a cluster Dedicated/LowPriority
ScalingType	Type of cluster scaling manual/auto
InitialNodeCount	Initial node count of the cluster
MinimumNodeCount	Minimum node count of the cluster
MaximumNodeCount	Maximum node count of the cluster
NodeDeallocationOption	How the node should be deallocated
Publisher	Publisher of the cluster type
Offer	Offer with which the cluster is created
Sku	Sku of the Node/VM created inside cluster
Version	Version of the image used while Node/VM is created
SubnetId	SubnetId of the cluster
AllocationState	Cluster allocation state
CurrentNodeCount	Current node count of the cluster
TargetNodeCount	Target node count of the cluster while scaling up/down
EventType	Type of event during cluster creation.
NodeIdleTimeSecondsBeforeScaleDown	Idle time in seconds before cluster is scaled down
PreemptedNodeCount	Preempted node count of the cluster
IsResizeGrow	Flag indicating that cluster is scaling up
VmFamilyName	Name of the VM family of the nodes that can be created inside cluster
LeavingNodeCount	Leaving node count of the cluster
UnusableNodeCount	Unusable node count of the cluster
IdleNodeCount	Idle node count of the cluster
RunningNodeCount	Running node count of the cluster
PreparingNodeCount	Preparing node count of the cluster
QuotaAllocated	Allocated quota to the cluster
QuotaUtilized	Utilized quota of the cluster
AllocationStateTransitionTime	Transition time from one state to another
ClusterErrorCodes	Error code received during cluster creation or scaling
CreationApiVersion	Api version used while creating the cluster

AmlComputeInstanceEvent table

Property	Description
Type	Name of the log event, AmlComputeInstanceEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
ResultType	The status of the event. Typical values include Started, In Progress, Succeeded, Failed, Active, and Resolved.
CorrelationId	A GUID used to group together a set of related events, when applicable.
OperationName	The name of the operation associated with the log entry
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
AmlComputeInstanceName	"The name of the compute instance associated with the log entry.

AmlDataLabelEvent table

Property	Description
Type	Name of the log event, AmlDataLabelEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
ResultType	The status of the event. Typical values include Started, In Progress, Succeeded, Failed, Active, and Resolved.
CorrelationId	A GUID used to group together a set of related events, when applicable.
OperationName	The name of the operation associated with the log entry
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
AmlProjectId	The unique identifier of the Azure Machine Learning project.
AmlProjectName	The name of the Azure Machine Learning project.
AmlLabelNames	The label class names which are created for the project.
AmlDataStoreName	The name of the data store where the project's data is stored.

AmlDataSetEvent table

Property	Description
Type	Name of the log event, AmlDataSetEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
ResultType	The status of the event. Typical values include Started, In Progress, Succeeded, Failed, Active, and Resolved.
AmlWorkspaceId	A GUID and unique ID of the Azure Machine Learning workspace.
OperationName	The name of the operation associated with the log entry
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
AmlDatasetId	The ID of the Azure Machine Learning Data Set.
AmlDatasetName	The name of the Azure Machine Learning Data Set.

AmlDataStoreEvent table

Property	Description
Type	Name of the log event, AmlDataStoreEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
ResultType	The status of the event. Typical values include Started, In Progress, Succeeded, Failed, Active, and Resolved.
AmlWorkspaceId	A GUID and unique ID of the Azure Machine Learning workspace.
OperationName	The name of the operation associated with the log entry
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
AmlDatastoreName	The name of the Azure Machine Learning Data Store.

AmlDeploymentEvent table

Property	Description
Type	Name of the log event, AmlDeploymentEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
ResultType	The status of the event. Typical values include Started, In Progress, Succeeded, Failed, Active, and Resolved.
OperationName	The name of the operation associated with the log entry
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
AmlServiceName	The name of the Azure Machine Learning Service.

AmlInferencingEvent table

Property	Description
Type	Name of the log event, AmlInferencingEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
ResultType	The status of the event. Typical values include Started, In Progress, Succeeded, Failed, Active, and Resolved.
OperationName	The name of the operation associated with the log entry
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
AmlServiceName	The name of the Azure Machine Learning Service.

AmlModelsEvent table

Property	Description
Type	Name of the log event, AmlModelsEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
ResultType	The status of the event. Typical values include Started, In Progress, Succeeded, Failed, Active, and Resolved.
OperationName	The name of the operation associated with the log entry
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
ResultSignature	The HTTP status code of the event. Typical values include 200, 201, 202 etc.
AmlModelName	The name of the Azure Machine Learning Model.

AmlPipelineEvent table

Property	Description
Type	Name of the log event, AmlPipelineEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
ResultType	The status of the event. Typical values include Started, In Progress, Succeeded, Failed, Active, and Resolved.
AmlWorkspaceId	A GUID and unique ID of the Azure Machine Learning workspace.
AmlWorkspaceId	The name of the Azure Machine Learning workspace.
OperationName	The name of the operation associated with the log entry
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
AmlModuleId	A GUID and unique ID of the module.
AmlModelName	The name of the Azure Machine Learning Model.
AmlPipelineId	The ID of the Azure Machine Learning pipeline.
AmlParentPipelineId	The ID of the parent Azure Machine Learning pipeline (in the case of cloning).
AmlPipelineDraftId	The ID of the Azure Machine Learning pipeline draft.
AmlPipelineDraftName	The name of the Azure Machine Learning pipeline draft.
AmlPipelineEndpointId	The ID of the Azure Machine Learning pipeline endpoint.
AmlPipelineEndpointName	The name of the Azure Machine Learning pipeline endpoint.

AmlRunEvent table

Property	Description
Type	Name of the log event, AmlRunEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
ResultType	The status of the event. Typical values include Started, In Progress, Succeeded, Failed, Active, and Resolved.
OperationName	The name of the operation associated with the log entry
AmlWorkspaceId	A GUID and unique ID of the Azure Machine Learning workspace.
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
RunId	The unique ID of the run.

AmlEnvironmentEvent table

Property	Description
Type	Name of the log event, AmlEnvironmentEvent
TimeGenerated	Time (UTC) when the log entry was generated
Level	The severity level of the event. Must be one of Informational, Warning, Error, or Critical.
OperationName	The name of the operation associated with the log entry
Identity	The identity of the user or application that performed the operation.
AadTenantId	The Microsoft Entra tenant ID the operation was submitted for.
AmlEnvironmentName	The name of the Azure Machine Learning environment configuration.
AmlEnvironmentVersion	The name of the Azure Machine Learning environment configuration version.

AMLOnlineEndpointTrafficLog table (preview)

Property	Description
Method	The method that the client requests.
Path	The path that the client requests.
SubscriptionId	The machine learning subscription ID of the online endpoint.
AzureMLWorkspaceId	The machine learning workspace ID of the online endpoint.
AzureMLWorkspaceName	The machine learning workspace name of the online endpoint.
EndpointName	The name of the online endpoint.
DeploymentName	The name of the online deployment.
Protocol	The protocol of the request.
ResponseCode	The final response code that's returned to the client.
ResponseCodeReason	The final response code reason that's returned to the client.
ModelStatusCode	The response status code from the model.
ModelStatusReason	The response status reason from the model.
RequestPayloadSize	The total bytes received from the client.
ResponsePayloadSize	The total bytes sent back to the client.
UserAgent	The user-agent header of the request, including comments but truncated to a maximum of 70 characters.
XRequestId	The request ID that Azure Machine Learning generates for internal tracing.
XMSClientRequestId	The tracking ID that the client generates.
TotalDurationMs	The duration in milliseconds from the request start time to the time the last response byte is sent back to the client. If the client disconnects, the duration is taken from the start time to the client disconnect time.
RequestDurationMs	The duration in milliseconds from the request start time to the time the last byte of the request is received from the client.
ResponseDurationMs	The duration in milliseconds from the request start time to the time the first response byte is read from the model.
RequestThrottlingDelayMs	The delay in milliseconds in the request data transfer due to network throttling.
ResponseThrottlingDelayMs	The delay in milliseconds in the response data transfer due to network throttling.

For more information on this log, see Monitor online endpoints.

AMLOnlineEndpointConsoleLog

Property	Description
TimeGenerated	The UTC time stamp of the time the log is generated
OperationName	The operation associated with the log record
InstanceId	The ID of the instance that generates the log record
DeploymentName	The name of the deployment associated with the log record
ContainerName	The name of the container where the log is generated
Message	The content of the log

For more information on this log, see Monitor online endpoints.

AMLOnlineEndpointEventLog (preview)

Property	Description
TimeGenerated	The UTC time stamp of the time the log is generated
OperationName	The operation associated with the log record
InstanceId	The ID of the instance that generates the log record
DeploymentName	The name of the deployment associated with the log record
Name	The name of the event
Message	The content of the event

For more information on this log, see Monitor online endpoints.

Related content

• See Monitor Machine Learning for a description of monitoring Machine Learning.
• See Monitor Azure resources with Azure Monitor for details on monitoring Azure resources.