Share via

MONITOR PIPELINE ACTIVITY

Tú Nguyễn 85 Reputation points
2024-07-09T08:57:29.4433333+00:00

Hi,

I'm working on monitoring the success or failure status of AML jobs on Log Analytics. However, I don't know which tables in Log Analytics contain logs of AML jobs. I followed this documentation (https://learn.microsoft.com/ms-my/azure/azure-monitor/reference/tables/amldatastoreevent) to find out the required table but I still do not know which table is correct.

Could someone help me to find the correct table?

Many thanks!

Azure Monitor
Azure Monitor
An Azure service that is used to collect, analyze, and act on telemetry data from Azure and on-premises environments.
3,025 questions
Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,720 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,932 questions
Azure Policy
Azure Policy
An Azure service that is used to implement corporate governance and standards at scale for Azure resources.
830 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Amira Bedhiafi 19,946 Reputation points
    2024-07-10T10:41:43.4133333+00:00

    The table you mentioned in the documentation (amldatastoreevent) is one of the tables related to AML, but there are other tables that you might find useful for tracking AML job statuses.

    1. AMLJobEvent: This table contains events related to Azure Machine Learning jobs. You can query this table to get information about the status, errors, and completion of your AML jobs. Example query: 
         
   AMLJobEvent
   
   | where JobStatus == "Failed" or JobStatus == "Completed"
   
   | project JobId, JobName, JobStatus, StartTime, EndTime, ErrorDetails
    2. AMLComputeClusterEvent: This table contains events related to compute clusters used by AML jobs. It can provide insights into the state of your compute resources. Example query: 
         
   AMLComputeClusterEvent
   
   | where ClusterState == "Failed" or ClusterState == "Succeeded"
   
   | project ClusterName, ClusterState, Timestamp, ErrorDetails
    3. AMLRunEvent: This table logs detailed events about individual runs within an AML experiment. You can track the progress and outcome of each run. Example query: 
         
   AMLRunEvent
   
   | where RunStatus == "Failed" or RunStatus == "Completed"
   
   | project RunId, ExperimentName, RunStatus, StartTime, EndTime, ErrorDetails
    4. AzureDiagnostics: This table may also contain relevant logs if you have configured diagnostic settings for your AML workspace to send logs to Azure Monitor. Example query: 
         
   AzureDiagnostics
   
   | where ResourceType == "Microsoft.MachineLearningServices/workspaces"
   
   | where OperationName == "RunStatusChanged"
   
   | project OperationName, Status, TimeGenerated, Resource