Hello,
I am seeking assistance regarding an issue with our Azure Monitor workbook (Key Metrics) where tiles designed to display top CPU-consuming processes per VM intermittently return “The query returned no results” — despite validated Perf and InsightsMetrics data being present.
Setup:
Workspace: Single Log Analytics workspace
Telemetry source: Azure VMs connected via DCR Connected VMs: AppServer2, AppServer3, WebServer3 (only this VM consistently returns data), and others
Workbook configuration:
- Auto-refresh: 5 min
- Query time window: Last 30 min
- Query: Join of
Perf
and InsightsMetrics
to compute normalized CPU
Example KQL (used in tiles)/ workbook:
Perf
| where ObjectName == "Process" and CounterName == "% Processor Time" and InstanceName != "_Total" and InstanceName != "Idle"
| join kind=inner (
InsightsMetrics
| where Namespace == "Processor"
| where Name == "UtilizationPercentage"
| where Computer contains "AppServer2"
| extend tagValues = parse_json(Tags)
| extend totalCpus = toint(tagValues["vm.azm.ms/totalCpus"])
| project TimeGenerated, Computer, TotalCPUs = totalCpus
) on Computer
| summarize MaxCPU = max(CounterValue) by Computer, TotalCPUs, bin(TimeGenerated, 1m), InstanceName
| extend NormalizedCPU = MaxCPU / (TotalCPUs * 100)
| summarize MaxNormalizedCPU = max(NormalizedCPU) by Computer, InstanceName
| order by MaxNormalizedCPU desc
| take 10
Tiles typically filter per VM (e.g., Computer == "AppServer2"
).
What we observed:
✅ Perf, InsightsMetrics, Heartbeat data are present
Perf
| where TimeGenerated > ago(1h)
| where ObjectName == "Process" and CounterName == "% Processor Time"
| summarize Count = count() by Computer
InsightsMetrics
| where TimeGenerated > ago(1h)
| where Namespace == "Processor" and Name == "UtilizationPercentage"
| extend tags = parse_json(Tags)
| extend TotalCPUs = toint(tags["vm.azm.ms/totalCpus"])
| where isnotnull(TotalCPUs)
| summarize by Computer, TotalCPUs
Heartbeat
| where TimeGenerated > ago(1h)
| summarize arg_max(TimeGenerated, ResourceId) by Computer
All VMs report data as expected in all three tables.
Naming discrepancies in Computer
field between tables
We noticed that Computer
values differ slightly between Perf
, InsightsMetrics
, and Heartbeat
.
- Example mappings:
-
DPSERVERl
→ /virtualMachines/DPSERVER-N
-
AppServer
→ /virtualMachines/AppServer2.xxx.xx.
-
WebServer3
→ /virtualMachines/WebServer3-ixxxxx.xxxx
This mismatch could impact joins when filtering by Computer
, especially at the tile level.
Join query works in Log Analytics: When run in Log Analytics, the join query returns expected results across all VMs.
Workbook tile-level filtering fails: When the same query is used in a workbook with per-VM tile filters (e.g., Computer == "AppServer2"
), most tiles return “The query returned no results” except for WebServer3.
What we tried:
Explicit binning on both tables (bin(TimeGenerated, 1m)
)
Introducing a VM map layer from Heartbeat
:
// Step 1: Map VMs
let VMMap = Heartbeat
| where TimeGenerated > ago(1h)
| summarize arg_max(TimeGenerated, ResourceId) by Computer
| extend VMName = extract(@"virtualMachines/([^/]+)", 1, ResourceId);
// Step 2: Process Perf Data
let PerfData = Perf
| where TimeGenerated > ago(1h)
| where ObjectName == "Process" and CounterName == "% Processor Time"
| where InstanceName !in ("_Total", "Idle")
| summarize MaxCPU = max(CounterValue) by Computer, InstanceName, TimeBin = bin(TimeGenerated, 1m);
// Step 3: CPU core data
let CPUData = InsightsMetrics
| where TimeGenerated > ago(1h)
| where Namespace == "Processor" and Name == "UtilizationPercentage"
| extend tags = parse_json(Tags)
| extend TotalCPUs = toint(tags["vm.azm.ms/totalCpus"])
| summarize TotalCPUs = max(TotalCPUs) by Computer, TimeBin = bin(TimeGenerated, 1m);
// Step 4: Join everything
PerfData
| join kind=inner (CPUData) on Computer, TimeBin
| join kind=leftouter (VMMap) on Computer
| extend NormalizedCPU = MaxCPU / (TotalCPUs * 100)
| project TimeBin, VMName, Computer, InstanceName, MaxCPU, TotalCPUs, NormalizedCPU
| order by NormalizedCPU desc
| take 20
This approach failed in the workbook (Failed to resolve table or column expression
).
Questions:
Could the join behavior for per-VM filtered workbook tiles behave differently from direct Log Analytics queries?
Could TimeGenerated
alignment (even with binning) still cause joins to fail at tile level?
Is this a known change or regression in workbook join behavior?
Is the Heartbeat + VMMap join pattern a recommended approach to align Perf + InsightsMetrics for tiles?
Are there better practices to design per-VM tiles (parameterized or static) that join Perf + InsightsMetrics reliably?
Does Microsoft recommend pre-aggregating or using a custom table instead of direct joins in this scenario?
Additional context:
DCR config verified (% Processor Time counter is enabled)
No data ingestion delay — data visible in both tables for the same window
The issue appears specific to workbook per-VM tiles