Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Microsoft Discovery CogLoop is the AI orchestration engine that drives investigation progress. Cognition Engine logs capture:
- Instance lifecycle - Cognition Engine instance start, stop, and polling activity
- Reasoning decisions - Thinking module (fast/slow) and acting module tool selections
- Task management operations - Task execution, validation, status transitions, and agent assignments
- Error diagnostics - Serialization failures, Cosmos DB connectivity issues, loop errors, and tool call failures
It continuously runs two subloops - Act and Cognition to plan and execute research tasks on your behalf. CogLoop logs are automatically stored in the DiscoveryCogLoopLogs_CL table in the Log Analytics workspace inside the workspace's Managed Resource Group (MRG).
Important
Check the table's plan and retention settings in the Log Analytics workspace without modifying managed infrastructure. For Basic or Auxiliary tables, use workspace scope and set an explicit time range in the Log Analytics picker or API timespan; a KQL time filter doesn't replace it. The available query window depends on the table's plan and configured retention. See Query Basic and Auxiliary tables and Manage data retention.
This article explains how to query CogLoop logs to monitor investigation activity, diagnose stalls, and investigate orchestration errors.
Prerequisites
Before you begin, navigate to the Log Analytics workspace in your workspace's MRG. For instructions, see Access resource logs for Microsoft Discovery resources.
Key identifiers
Before writing queries, familiarize yourself with the following identifiers:
| Identifier | Description | Where to find it |
|---|---|---|
InstanceId |
ID for an investigation's CogLoop instance, such as cog:{project}:{investigation}-{shortHash} |
Retrieve the full value from logs by using List all Cognition Engine instances; don't guess the generated suffix. |
ModuleName |
Which subloop generated the log: Act or Cognition |
Extracted from the Properties JSON field |
CorrelationId |
Identifier for related application log entries, when populated | Read the value from matching log entries; don't assume it matches an Activity Log correlationId or an API response-header value. |
Query CogLoop logs
After opening the Logs interface in the workspace's MRG Log Analytics workspace:
- In the left panel, select the Tables tab.
- Expand Custom Logs and locate
DiscoveryCogLoopLogs_CL. - Run the View recent logs query to check whether data is available in the selected time range.
Log schema
The examples use the following fields and properties in DiscoveryCogLoopLogs_CL. Expand the table in the Tables pane to inspect its columns, then inspect recent rows and Properties values. Adapt the queries if the available fields, properties, or message formats differ.
| Field Name | Description |
|---|---|
TimeGenerated |
Timestamp used by Log Analytics for time filtering |
TimeStamp |
Application-provided timestamp; inspect populated values before relying on its precision |
LogLevel |
Log severity level: Debug, Information, Warning, or Error |
Message |
Primary log message content |
Exception |
Full exception details including stack trace (populated on error entries) |
Properties |
JSON object containing structured properties such as InstanceId, ChosenTool, ModuleName, ClassName, MethodName, and others |
CorrelationId |
Identifier for related application log entries, when populated |
TenantId |
Workspace ID in the Azure Monitor standard schema; verify this custom table's definition before using it as a tenant filter |
Key Properties (inside the Properties JSON field)
| Property | Description |
|---|---|
InstanceId |
Cognition Engine instance identifier. Retrieve the full value from logs, including any generated suffix. |
ModuleName |
Reasoning module name (Cognition or Act) |
ChosenTool |
The tool/function selected by the PickBest decision engine |
ClassName |
Source class name (for example, CogLoopInstanceManager, CosmosDbService) |
MethodName |
Source method name |
Goal |
The reasoning prompt goal submitted to the PickBest engine |
SleepTime |
Wait duration in seconds when the Cognition Engine decides to wait |
TaskName |
Task identifier for task management operations |
Attempt |
Retry attempt number for validation operations |
Result |
Tool execution result summary |
Example queries
View recent logs
DiscoveryCogLoopLogs_CL
| order by TimeGenerated desc
| take 100
Filter by time range
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(1h)
| order by TimeGenerated desc
View all errors and warnings
Find logged CogLoop errors and warnings in the selected time range. Then, compare them with task status to investigate a session.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(1h)
| where LogLevel in ("Error", "Warning")
| project TimeGenerated, LogLevel, Message
| order by TimeGenerated desc
Adjust ago(1h) to ago(6h), ago(1d), and so on, as needed.
Log Volume by Level Over Time
Charts the volume of Cognition Engine log entries per hour by severity level. Each log level renders as a separate line, making it easy to spot error spikes.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| summarize Count = count() by LogLevel, bin(TimeGenerated, 1h)
| order by TimeGenerated desc
| render timechart
An error spike identifies a period to investigate. Compare it with task status and other logs before concluding that the service failed or recovered.
List All Cognition Engine Instances
Lists Cognition Engine instances (investigations) found in the selected log time range and classifies their recorded activity into three operation types.
- InstanceId - Full CogLoop instance ID found in the logs, including any generated suffix.
- StatusChecks - Number of matching status-check log entries; compare instances within the same time range.
- Starts - Number of log entries mentioning
StartInstanceAsync; inspect the entries and task status to confirm startup succeeded. - Retrievals - Number of log entries mentioning
GetRunningInstanceAsync; inspect the messages for state retrieval results.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where Message has "IsInstanceRunningAsync" or Message has "StartInstanceAsync" or Message has "GetRunningInstanceAsync"
| extend
InstanceId = tostring(parse_json(Properties).InstanceId),
Operation = case(
Message has "StartInstanceAsync", "Started",
Message has "GetRunningInstanceAsync", "Retrieved",
"StatusCheck")
| where isnotempty(InstanceId)
| summarize
LastSeen = max(TimeGenerated),
StatusChecks = countif(Operation == "StatusCheck"),
Starts = countif(Operation == "Started"),
Retrievals = countif(Operation == "Retrieved")
by InstanceId
| order by Starts desc, Retrievals desc, LastSeen desc
The sort order puts instances with logged start or retrieval activity at the top. Confirm current status in Discovery Studio rather than treating these counts as proof that an investigation is running.
Check whether a specific CogLoop instance is running
Verify whether CogLoop is actively monitoring your investigation. Replace <your-project> with your project name.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(15m)
| where Message has "IsInstanceRunningAsync"
| extend InstanceId = extract(@"InstanceId: (cog:[^\s]+)", 1, Message)
| where InstanceId has "<your-project>"
| summarize last_checked = max(TimeGenerated) by InstanceId
| order by last_checked desc
A recent last_checked shows that polling was logged. No results can reflect the selected filters, log availability, or ingestion delay; it doesn't prove that CogLoop stopped monitoring the investigation.
List all active investigations being monitored
See which investigations CogLoop is currently tracking, useful when you want to confirm whether your investigation is in scope.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(30m)
| where Message has "IsInstanceRunningAsync"
| extend InstanceId = extract(@"InstanceId: (cog:[^\s]+)", 1, Message)
| where isnotempty(InstanceId)
| summarize last_seen = max(TimeGenerated) by InstanceId
| order by last_seen desc
Track Instance Startup
Retrieve logged events for a specific Cognition Engine instance in chronological order within the selected time range. Replace <your-instance-id> with the full instance ID retrieved from logs.
- TimeGenerated - When the event occurred, sorted oldest-first to reconstruct the sequence of events.
- LogLevel - Severity level, useful for spotting where errors or warnings interrupted the instance lifecycle.
- Message - The log message content, showing startup steps, reasoning decisions, task operations, and any failures in order.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| extend InstanceId = tostring(parse_json(Properties).InstanceId)
| where InstanceId == "<your-instance-id>"
| project TimeGenerated, LogLevel, Message
| order by TimeGenerated asc
This query helps you review a specific investigation's recorded behavior and identify entries to investigate alongside its current task status.
View all activity for a specific investigation
Retrieve available CogLoop activity log entries for one investigation within the selected time range. Replace cog:<your-project>:<your-investigation> with the InstanceId prefix found in your logs.
DiscoveryCogLoopLogs_CL
| where Message has "cog:<your-project>:<your-investigation>"
| project TimeGenerated, LogLevel, Message
| order by TimeGenerated asc
View errors for a specific investigation
View all errors and warnings for a single investigation instance.
DiscoveryCogLoopLogs_CL
| where LogLevel in ("Error", "Warning")
| where Message has "cog:<your-project>:<your-investigation>"
or Exception has "cog:<your-project>:<your-investigation>"
| project TimeGenerated, LogLevel, Message, Exception
| order by TimeGenerated asc
Diagnose why CogLoop isn't making progress
CogLoop enters a wait loop when it's blocked, typically waiting for a running task to complete or for human intervention on a flagged task.
DiscoveryCogLoopLogs_CL
| where Message has_any ("No task updates", "Skipping redundant action", "Cognition-Wait")
| extend InstanceId = extract(@"InstanceId: (cog:[^\s]+)", 1, Message)
| extend SleepTime = toint(extract(@"wait period of (\d+) seconds", 1, Message))
| project TimeGenerated, LogLevel, InstanceId, SleepTime, Message
| order by TimeGenerated desc
Cognition Reasoning Loop
Detect Cognition Wait Loops
If the Cognition Engine repeatedly selects Cognition-Wait, it can mean tasks are stalled, all work is already complete, or an internal error is preventing progress.
- IdlePct - Percentage of logged waits marked
NO UPDATE. Sustained 100% is a reason to check task state, not proof that the investigation is stuck. - AvgSleepSec - Average logged wait duration. A longer wait doesn't by itself mean that cognition stopped trying.
- SampleReason - The LLM's own explanation
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(6h)
| where Message has "PickBest Result" and Message has "Cognition-Wait"
| extend
WaitResult = iff(Message has "NO UPDATE", "Idle", "Progress"),
SleepTime = toint(extract(@"sleepTime[^\d]*(\d+)", 1, Message)),
Reasoning = extract(@""""Reasoning"""":\s*""""([^""""]+)", 1, Message)
| summarize
Waits = count(),
IdlePct = round(100.0 * countif(WaitResult == "Idle") / count(), 0),
AvgSleepSec = round(avg(SleepTime), 0),
SampleReason = take_any(Reasoning)
by bin(TimeGenerated, 30m)
| where Waits > 3
| order by TimeGenerated desc
This query groups waits into 30-minute windows and surfaces whether cognition is idle or making progress.
View Slow and Fast Thinking Activity
Trace the reasoning steps the Cognition Engine takes before acting. Slow thinking indicates complex deliberation; fast thinking indicates straightforward decisions.
- TimeGenerated - Timestamp of the matching log entry.
- ThinkingType -
FastThinkingorSlowThinking, indicating the depth of reasoning applied. - Message - The full thinking output, including the thought content and reasoning context.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(6h)
| where Message has "thinking completed for thought"
| extend ThinkingType = iff(Message has "Slow thinking", "SlowThinking", "FastThinking")
| project TimeGenerated, ThinkingType, Message
| order by TimeGenerated desc
If
SlowThinkingentries dominate, the Cognition Engine is spending significant effort on complex decisions, this may be expected for difficult investigations or could indicate unclear task definitions forcing repeated deep analysis.
Task Management Operations
Track Task Execution Lifecycle
Retrieve logged events for a specific task in chronological order, including assignment, agent selection, execution, validation, and completion when recorded. Replace <your-task-id> with the target task name.
let taskId = "<your-task-id>";
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where Message contains taskId
| project TimeGenerated, LogLevel, Message
| order by TimeGenerated asc
Use these entries to investigate where a task might have stalled or failed. Confirm its current status in Discovery Studio; missing log entries don't establish completion or failure.
View Task Validation Results
List matching task validation log entries, showing when and how the Cognition Engine evaluated task results. Inspect the Message field for validation outcomes such as Complete, Incomplete, Needs User Attention, or Failed.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where Message has "TASKMANAGEMENT: tool call: ValidateTask"
| project TimeGenerated, Message
| order by TimeGenerated desc
Useful for understanding why a task was marked complete or sent back for rework. If you see repeated
Incompleteresults for the same task, it may indicate the assigned agent is not producing satisfactory output, or the validation criteria are too strict.
View TaskValidationAgent Lifecycle
Find logged TaskValidationAgent events, including provisioning, upsert, invocation, and completion when recorded. Use these entries to investigate whether the validation agent was created and invoked.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where Message has "TaskValidationAgent"
| project TimeGenerated, LogLevel, Message
| order by TimeGenerated asc
If no entries appear, first rule out missing or delayed logs. If entries show upsert or invocation errors, verify the validation agent and the model deployment configured for it. See Discovery Engine startup checks.
Error Diagnostics
View All Errors with Exceptions
List all error entries with their full exception details for root-cause analysis.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where LogLevel == "Error"
| project TimeGenerated, Message, Exception
| order by TimeGenerated desc
Analyze Error Patterns
Summarize errors by message to identify the most frequent failure modes.
- ErrorMessage - The first 80 characters of the error message, used as a grouping key to cluster similar errors together.
- ErrorCount - How many times each error was logged. The highest counts identify the most frequent logged errors, not necessarily those with the greatest impact.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where LogLevel == "Error"
| summarize ErrorCount = count() by ErrorMessage = substring(Message, 0, 80)
| order by ErrorCount desc
Start with frequent error types, then compare their timestamps and instance IDs with related errors before assigning a root cause.
Detect Cosmos DB Connectivity Issues
Cosmos DB health failures can prevent the Cognition Engine from retrieving instance state and stall affected investigations.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where Message has "Cosmos DB health check failed" or Message has "Cosmos DB is unreachable"
| summarize FailureCount = count() by bin(TimeGenerated, 15m)
| order by TimeGenerated desc
| render timechart
Detect Serialization Errors (JsonException)
Isolates JSON serialization errors and groups them by message, including a sample stack trace for each. If an instance's state can't be deserialized, its reasoning activity can be blocked.
- ErrorMessage - The first 80 characters of the error message, grouping related serialization failures together.
- Count - How many times each serialization error occurred. High counts show recurrence in the selected time range; compare affected instances to assess the scope.
- SampleException - A full exception with stack trace, showing the exact JSON path and property that failed to deserialize (for example,
AuthorRole).
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where LogLevel == "Error"
| where Exception has "JsonException"
| summarize
Count = count(),
SampleException = take_any(Exception)
by ErrorMessage = substring(Message, 0, 80)
| order by Count desc
Use SampleException to identify the JSON path or property that couldn't be deserialized. Compare related instance and polling errors before assigning a cause; the log pattern alone doesn't establish that a service upgrade caused the failure.
Detect Polling Cycle Failures
Polling cycle errors prevent the Cognition Engine from discovering and starting new work.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where Message has "Error during polling cycle"
| project TimeGenerated, Message, Exception
| order by TimeGenerated desc
Error Timeline for Incident Investigation
Correlate errors over time to identify when an incident started and whether it's ongoing.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where LogLevel == "Error"
| summarize ErrorCount = count() by bin(TimeGenerated, 5m)
| order by TimeGenerated desc
| render timechart
Use this query to identify periods with logged errors. Pair it with the Analyze Error Patterns query and current task status; a drop in logged errors alone doesn't prove recovery.
Health and Service Status
Detect Unhealthy Periods
Find time windows where health checks reported unhealthy status, which correlates with service disruptions.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where Message has "Health check" and Message has "Unhealthy"
| extend
HealthCheckName = tostring(parse_json(Properties).HealthCheckName),
Description = tostring(parse_json(Properties).HealthCheckDescription)
| summarize UnhealthyCount = count() by HealthCheckName, bin(TimeGenerated, 15m)
| order by TimeGenerated desc
| render timechart
Monitor Instance Retrieval Errors
Repeated failures in instance retrieval indicate the service can't load investigation state from Cosmos DB.
DiscoveryCogLoopLogs_CL
| where TimeGenerated > ago(24h)
| where Message has "Error retrieving all cogloop instances"
| summarize FailureCount = count() by bin(TimeGenerated, 15m)
| order by TimeGenerated desc
| render timechart
What to look for in the results:
| Pattern | Meaning |
|---|---|
Repeated No task updates during wait period of 60 seconds |
CogLoop is waiting for a running task to finish; check the associated Act task for status |
Skipping redundant action: Cognition-Wait with result: NO UPDATE |
Cognition is also blocked; no new decisions are being made |
| Pattern persists for hours | A task may be FLAGGED_HUMAN and requires your review in Discovery Studio |
View CogLoop decision log for an investigation
View the reasoning behind CogLoop's recent decisions, which tool it chose and why. Useful to understand why CogLoop is repeatedly waiting instead of making progress.
DiscoveryCogLoopLogs_CL
| where Message has "PickBest"
| where Message has "cog:<your-project>:<your-investigation>"
or CorrelationId == "<your-correlationId>"
| extend chosen_tool = extract(@'"ChosenTool":\s*"([^"]+)"', 1, Message)
| extend confidence = toint(extract(@'"Confidence":\s*(\d+)', 1, Message))
| extend reasoning = extract(@'"Reasoning":\s*"([^"]+)"', 1, Message)
| project TimeGenerated, chosen_tool, confidence, reasoning
| order by TimeGenerated desc
The reasoning field explains exactly why CogLoop chose to wait or act, it's the most useful field for understanding investigation stalls.
Detect Act and Cognition subloop errors
The two CogLoop subloops can fail independently. This query breaks down errors by subloop to identify which one is failing.
DiscoveryCogLoopLogs_CL
| where LogLevel == "Error"
| where Message has_any ("Act loop error", "Cognition loop error")
| extend ModuleName = case(
Message has "Act loop", "Act",
Message has "Cognition loop", "Cognition",
"Unknown"
)
| summarize error_count = count() by ModuleName, bin(TimeGenerated, 1h)
| order by TimeGenerated desc
- Act loop errors affect task execution and tool invocation.
- Cognition loop errors affect planning and decision-making.
- Errors in both loops simultaneously may indicate a systemic issue such as a service outage or network partition.
Detect context window saturation
When CogLoop's context window exceeds 80% capacity, it attempts to reset working memory. If that reset also fails because the context is too large, an error is logged.
DiscoveryCogLoopLogs_CL
| where Message has_any ("Context window saturation", "working memory context reset")
or (LogLevel == "Error" and Message == "Error during working memory context reset")
| extend event_type = case(
Message has "80%", "Warning: Context window near full",
Message == "Error during working memory context reset", "Error: Reset failed",
"Other"
)
| project TimeGenerated, LogLevel, event_type, Message, Exception
| order by TimeGenerated asc
Note
This error indicates the investigation has accumulated a large context. CogLoop attempted to compress it but the summary was too large for the model's input limit. If this error repeats without recovery, the investigation may be stuck. Contact your Discovery administrator.
Detect circuit breaker events
A circuit breaker opening means that repeated LLM API call failures caused CogLoop to pause sending requests temporarily to prevent cascading failures.
DiscoveryCogLoopLogs_CL
| where LogLevel == "Error"
| where Exception has "BrokenCircuitException"
| project TimeGenerated, LogLevel, Message, Exception
| order by TimeGenerated desc
Note
Circuit breaker events typically self-heal when the circuit resets after a cooldown period. If they persist, contact Microsoft Support.
Troubleshooting
No data in DiscoveryCogLoopLogs_CL
| Cause | Resolution |
|---|---|
| Workspace was recently created and no investigations have run | Run an investigation to generate log entries |
| Time range is too narrow | Expand the time range to the last 24 hours |
| Logs are delayed due to ingestion latency | Allow for ingestion latency and rerun the query. See Log ingestion latency. |
Investigation appears stuck with no progress
| Cause | Resolution |
|---|---|
A task is in FLAGGED_HUMAN state |
Open the investigation in Discovery Studio and address the flagged task |
| CogLoop is waiting for a running tool task to complete | Check Supercomputer logs for the associated job. See Query supercomputer logs |
| Context window saturation reset failed | Contact your Discovery administrator |
Serialization Errors Blocking All Instances
Possible Causes:
- A Cognition Engine instance has working memory state that can't be deserialized (for example, after a service upgrade that changes model schemas)
- Corrupted instance data in Cosmos DB
Resolution:
- Run the Detect Serialization Errors query to confirm the error pattern
- Look at the
Exceptionfield for the specific JSON path and property that fails - Escalate to the service team with the instance ID and exception details
Tasks Not Being Validated
Possible Causes:
- TaskValidationAgent isn't deployed in the workspace
- The validation agent's configured model deployment isn't available.
Resolution:
- Run the View TaskValidationAgent Lifecycle query to check if the agent was created
- Look for upsert failures or provisioning errors
- Verify the required model deployment exists in the workspace
Query timeout or slow performance
| Cause | Resolution |
|---|---|
| Query scope is too broad | Narrow the time range or filter by InstanceId |
| Large dataset with no filters | Add filters on LogLevel, InstanceId, or Message before aggregating |