Azure Workbook Tile Returns “Query Returned No Results” for VMs Despite Perf & InsightsMetrics Data Being PresentAzure Workbook

Question

Azure Workbook Tile Returns “Query Returned No Results” for VMs Despite Perf & InsightsMetrics Data Being PresentAzure Workbook

Niket Kumar Singh 695

issue with an Azure Monitor shared dashboard (Resource-Monitor-v4) that contains workbook tiles displaying Top 10 CPU-consuming processes per VM using a KQL join between Perf and InsightsMetrics. This setup was previously working fine for all VMs, but recently only one VM (WebServer3) consistently returns data, while all others (AppServer2, AppSer, etc.)

show:The query returned no results.

This behavior is breaking our visibility into VM-level performance. We’ve validated everything from query structure, agent connectivity, data flow, and DCR, yet the issue persists.

Details of the Setup

Workspace: Single Log Analytics workspace
Telemetry source: Azure VMs connected via DCR (Data Collection Rules)
Connected VMs:
- AppServer2
- AppServ...etc
- WebServ (Only VM returning data)
- And several others

Workbook Configuration:

Auto-refresh: 5 minutes
Query time window: Last 30 minutes
Query aggregates % Processor Time from Perf table

Joins with CPU count from InsightsMetrics to normalize by core count. Original Working KQL Query (Still Works Only for One VM):

    Perf
    | where ObjectName == "Process" and CounterName == "% Processor Time" and InstanceName != "_Total" and InstanceName != "Idle"
    | join kind=inner (
        InsightsMetrics
        | where Namespace == "Processor"
        | where Name == "UtilizationPercentage"
        | where Computer contains "AppServer2"
        | extend tagValues = parse_json(Tags)
        | extend totalCpus = toint(tagValues["vm.azm.ms/totalCpus"])
        | project TimeGenerated, Computer, TotalCPUs = totalCpus
    ) on Computer
    | summarize MaxCPU = max(CounterValue) by Computer, TotalCPUs, bin(TimeGenerated, 1m), InstanceName
    | extend NormalizedCPU = MaxCPU / (TotalCPUs * 100)
    | summarize MaxNormalizedCPU = max(NormalizedCPU) by Computer, InstanceName
    | order by MaxNormalizedCPU desc
    | take 10

Observations and Diagnostics Heartbeat Logs:

All affected VMs are connected and sending Heartbeat logs to the workspace.
Verified in Log Analytics: Heartbeat | summarize by Computer – lists all expected machines.

Performance Data is Available:

Perf
| where TimeGenerated > ago(1h)
| where ObjectName == "Process" and CounterName == "% Processor Time"
| summarize Count = count() by Computer

CPU Info is Available in InsightsMetrics:

InsightsMetrics
| where TimeGenerated > ago(1h)
| where Namespace == "Processor" and Name == "UtilizationPercentage"
| extend tagValues = parse_json(Tags)
| extend TotalCPUs = toint(tagValues["vm.azm.ms/totalCpus"])
| where isnotnull(TotalCPUs)
| summarize by Computer, TotalCPUs

Workbook Visual Impact

Tiles for AppServer2, AppSer, AppSeN, etc., all show: “The query returned no results.”

as the behavior of KQL join changed in Log Analytics?
Could this be a regression or backend ingestion delay?
Is this related to subtle mismatches in how TimeGenerated alignment works between Perf and InsightsMetrics?
Are there any known issues or changes in Log Analytics join behavior, sampling delays, or InsightsMetrics ingestion?
Is there a more reliable way to join Perf with InsightsMetrics for normalized CPU data

Any insights would be greatly appreciated. We've done extensive testing and are trying to restore full visibility into our VM CPU behavior via the dashboard.

Vinod Pittala 3,695 Reputation points Microsoft External Staff Moderator

2025-06-13T05:24:22.75+00:00
Hello Niket Kumar Singh,

Thank you for providing detailed information about your Azure Monitor dashboard issue. Based on your description, it appears that the problem may be related to several factors, including query alignment, DCR settings, data ingestion latency, and potential changes in Azure Monitor's behavior. Below are some insights and recommendations to help you troubleshoot and resolve the issue.

Firstly, ensure the data collection rule (DCR) while enabling the data sources, that is you have to select the performance counters for the selected VM's.

The Perf and InsightsMetrics tables may have different data collection intervals, leading to misalignment when performing a join. This misalignment can result in missing data for certain VMs. So, use explicit time binning to align the data from both tables. For example:
Perf | where ObjectName == "Process" and CounterName == "% Processor Time" and InstanceName != "_Total" and InstanceName != "Idle" | summarize MaxCPU = max(CounterValue) by Computer, InstanceName, bin(TimeGenerated, 1m) | join kind=inner ( InsightsMetrics | where Namespace == "Processor" and Name == "UtilizationPercentage" | extend tagValues = parse_json(Tags) | extend TotalCPUs = toint(tagValues["vm.azm.ms/totalCpus"]) | summarize by Computer, TotalCPUs, bin(TimeGenerated, 1m) ) on Computer, TimeGenerated | extend NormalizedCPU = MaxCPU / (TotalCPUs * 100) | summarize MaxNormalizedCPU = max(NormalizedCPU) by Computer, InstanceName | order by MaxNormalizedCPU desc | take 10
This approach ensures that both tables are aligned to the same time bins, reducing the chances of missing data due to timing discrepancies.

Azure Monitor typically experiences a data ingestion latency of 20 seconds to 3 minutes. However, this can vary based on factors such as agent upload frequency, network conditions, and Azure platform processing times. So, Adjust your query's time window to account for potential ingestion delays. For instance, if your query is set to the last 30 minutes, consider extending it to the last 60 minutes to capture any delayed data. refer the doc: https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-ingestion-time?utm_source=chatgpt.com

Check that all affected VMs are correctly configured with the appropriate DCRs and agents.

Try the above recommendations and let me know if you have any other concerns.

If the provided solution works for your query, please do not forget to Upvote it. this can be beneficial to other community members.it would be greatly appreciated and helpful to others.

Thanks
Niket Kumar Singh 695 Reputation points

2025-06-13T12:13:01.0166667+00:00
Hi Vinod Pittala

Thank you for your detailed response and suggested query adjustments.

We did test the updated KQL query with explicit bin(TimeGenerated, 1m) on both the Perf and InsightsMetrics tables as recommended. While this modification successfully returns results when the query is run independently, and even within a shared workbook tile aggregating across all VMs, our specific use case involves one tile per VM, each configured with a Computer == "<VMName>" parameter.

In that context:

The same revised query shows "The query returned no results" for all tiles except one VM (WebServer3).

All other VM tiles (e.g., AppServer2, AppServer3, AppServer2-ISS-N) continue showing no output despite verified data in both Perf and InsightsMetrics.

We've validated that:

All VMs are sending Heartbeat, Perf, and InsightsMetrics data to the workspace.

The DCR configuration is correct and includes the % Processor Time counter under Process object.

The TotalCPU tag (vm.azm.ms/totalCpus) is present in InsightsMetrics for each VM.

There is no data ingestion delay issue—the Perf and InsightsMetrics data is visible for the same time range.

This issue appears to be specific to tile-level filtering per VM in a workbook using this JOIN, rather than the query logic itself.

Let me know if:

There are additional adjustments or JOIN strategies you recommend for parameterized tiles, or

You suspect this might relate to recent changes in how TimeGenerated is handled in joins across tables.

Thanks again for your guidance so far—we appreciate the insights and are keen to resolve this while maintaining our per-VM tile layout.
Niket Kumar Singh 695 Reputation points

2025-06-16T18:08:40.1666667+00:00
Hello Vinod Pittala,

I’m following up on the previously reported issue regarding our Azure Monitor workbook (Key Metrics-top) where tiles designed to display Top CPU-consuming processes per VM intermittently return “The query returned no results”, despite validated Perf and InsightsMetrics data.

New Observations Since Last Post

We’ve conducted extensive additional validation and cross-checked across different layers. Here’s a consolidated view:

Perf data confirmed:

Perf | where TimeGenerated > ago(1h) | where ObjectName == "Process" and CounterName == "% Processor Time" | summarize Count = count() by Computer

Returns valid process metrics across all VMs (e.g., AppServer2, AppServer3, etc.)

InsightsMetrics data confirmed:

InsightsMetrics | where TimeGenerated > ago(1h) | where Namespace == "Processor" and Name == "UtilizationPercentage" | extend tagValues = parse_json(Tags) | extend TotalCPUs = toint(tagValues["vm.azm.ms/totalCpus"]) | where isnotnull(TotalCPUs) | summarize by Computer, TotalCPUs

Returns expected CPU core count across all VMs.

Heartbeat data confirmed:

Heartbeat | where TimeGenerated > ago(1h) | summarize arg_max(TimeGenerated, ResourceId) by Computer

All VMs show recent Heartbeat + valid ResourceId mapping.

JOIN query in Log Analytics works: When run manually in Log Analytics, our join query combining Perf and InsightsMetrics returns normalized CPU as expected:

Perf | where TimeGenerated > ago(1h) and ObjectName == "Process" and CounterName == "% Processor Time" and InstanceName !in ("_Total", "Idle") | extend VMName = extract(@"virtualMachines/([^/]+)", 1, _ResourceId) | summarize MaxCPU = max(CounterValue) by _ResourceId, VMName, Computer, InstanceName, bin(TimeGenerated, 1m) | join kind=inner ( InsightsMetrics | where TimeGenerated > ago(1h) | where Namespace == "Processor" and Name == "UtilizationPercentage" | extend tags = parse_json(Tags) | extend TotalCPUs = toint(tags["vm.azm.ms/totalCpus"]) | extend VMName = extract(@"virtualMachines/([^/]+)", 1, _ResourceId) | summarize TotalCPUs = max(TotalCPUs) by _ResourceId, VMName, bin(TimeGenerated, 1m) ) on _ResourceId, TimeGenerated | extend NormalizedCPU = MaxCPU / (TotalCPUs * 100) | project TimeGenerated, VMName, Computer, InstanceName, NormalizedCPU | order by NormalizedCPU desc | take 4000

Results include all VMs (not just one).

However in Workbook (Key Metrics-top-10):

Tiles are based on a similar join query but filtered per VM (e.g., Computer == "AppServer2" via parameter or hardcoded filter).

Despite valid data, tiles display: “The query returned no results”

Only one VM tile (e.g., WebServer3) consistently shows data.

Suspicions / Points Needing Clarification

Could the join behavior in workbook tiles with per-VM filtering behave differently than in direct Log Analytics runs?
Could TimeGenerated alignment issues (despite binning) be causing this at workbook tile level? Are there known regressions or changes in Azure Monitor workbook joins (Perf + InsightsMetrics) that affect per-VM tile queries?
Is there a recommended pattern for reliably joining Perf and InsightsMetrics in parameterized workbook tiles to avoid this issue?

Add VMMap Layer for Mapping

To improve alignment and possibly address the issue, we propose introducing a VM map layer using Heartbeat → ResourceId → VMName extraction:

// Step 1: Map VMs let VMMap = Heartbeat | where TimeGenerated > ago(1h) | summarize arg_max(TimeGenerated, ResourceId) by Computer | extend VMName = extract(@"virtualMachines/([^/]+)", 1, ResourceId); // Step 2: Process Perf Data let PerfData = Perf | where TimeGenerated > ago(1h) | where ObjectName == "Process" and CounterName == "% Processor Time" | where InstanceName !in ("_Total", "Idle") | summarize MaxCPU = max(CounterValue) by Computer, InstanceName, TimeBin = bin(TimeGenerated, 1m); // Step 3: CPU core data let CPUData = InsightsMetrics | where TimeGenerated > ago(1h) | where Namespace == "Processor" and Name == "UtilizationPercentage" | extend tags = parse_json(Tags) | extend TotalCPUs = toint(tags["vm.azm.ms/totalCpus"]) | summarize TotalCPUs = max(TotalCPUs) by Computer, TimeBin = bin(TimeGenerated, 1m); // Step 4: Join everything PerfData | join kind=inner (CPUData) on Computer, TimeBin | join kind=leftouter (VMMap) on Computer | extend NormalizedCPU = MaxCPU / (TotalCPUs * 100) | project TimeBin, VMName, Computer, InstanceName, MaxCPU, TotalCPUs, NormalizedCPU | order by NormalizedCPU desc | take 50

'join' operator: Failed to resolve table or column expression named 'PerfData' Request id: 30f20dcf-7e31-4401-b1b8-f144b115423d

Is this additional VMMap join a recommended or reliable approach to ensure alignment of Perf + InsightsMetrics + VM identity for workbook tiles?

Could this help resolve the issue where tiles with per-VM filtering (via parameters or static filter) fail despite valid data?

Are there better practices for joining Perf + InsightsMetrics in Azure Monitor workbooks where per-VM tiles are used?

Is there any known change in join behavior at workbook level that could explain this disparity (Log Analytics query working, workbook tile failing)?

Does Microsoft recommend a different strategy (e.g., pre-aggregating into a custom table, union instead of join)?
Mallikarjuna Vardham 450 Reputation points Microsoft External Staff Moderator

2025-06-17T15:38:47.21+00:00

Hello Niket Kumar Singh,

Could you please check Make sure both Perf and InsightsMetrics data are binned using bin(TimeGenerated, 1m) before the join this helps align the timestamps properly.
Mallikarjuna Vardham 450 Reputation points Microsoft External Staff Moderator

2025-06-18T16:51:59.1766667+00:00

Hello Niket Kumar Singh,

just checking if you had a chance to look at my suggestion
Niket Kumar Singh 695 Reputation points

2025-06-24T10:07:13.9166667+00:00

Hi Mallikarjuna Vardham

Thanks for the guidence, we achived the goal, please close this thread.

Your answer

Niket Kumar Singh 695 Reputation points

2025-06-13T12:13:01.0166667+00:00

Hi Vinod Pittala

Thank you for your detailed response and suggested query adjustments.

We did test the updated KQL query with explicit bin(TimeGenerated, 1m) on both the Perf and InsightsMetrics tables as recommended. While this modification successfully returns results when the query is run independently, and even within a shared workbook tile aggregating across all VMs, our specific use case involves one tile per VM, each configured with a Computer == "<VMName>" parameter.

In that context:

The same revised query shows "The query returned no results" for all tiles except one VM (WebServer3).

All other VM tiles (e.g., AppServer2, AppServer3, AppServer2-ISS-N) continue showing no output despite verified data in both Perf and InsightsMetrics.

We've validated that:

All VMs are sending Heartbeat, Perf, and InsightsMetrics data to the workspace.

The DCR configuration is correct and includes the % Processor Time counter under Process object.

The TotalCPU tag (vm.azm.ms/totalCpus) is present in InsightsMetrics for each VM.

There is no data ingestion delay issue—the Perf and InsightsMetrics data is visible for the same time range.

This issue appears to be specific to tile-level filtering per VM in a workbook using this JOIN, rather than the query logic itself.

Let me know if:

There are additional adjustments or JOIN strategies you recommend for parameterized tiles, or

You suspect this might relate to recent changes in how TimeGenerated is handled in joins across tables.

Thanks again for your guidance so far—we appreciate the insights and are keen to resolve this while maintaining our per-VM tile layout.
Mallikarjuna Vardham 450 Reputation points Microsoft External Staff Moderator

2025-06-17T15:38:47.21+00:00

Hello Niket Kumar Singh,

Could you please check Make sure both Perf and InsightsMetrics data are binned using bin(TimeGenerated, 1m) before the join this helps align the timestamps properly.
Mallikarjuna Vardham 450 Reputation points Microsoft External Staff Moderator

2025-06-18T16:51:59.1766667+00:00

Hello Niket Kumar Singh,

just checking if you had a chance to look at my suggestion
Niket Kumar Singh 695 Reputation points

2025-06-24T10:07:13.9166667+00:00

Hi Mallikarjuna Vardham

Thanks for the guidence, we achived the goal, please close this thread.

Share via

Azure Workbook Tile Returns “Query Returned No Results” for VMs Despite Perf & InsightsMetrics Data Being PresentAzure Workbook

Your answer