Data Drift feature histogram chart shows wrong values against category labels for categorical features (label/value order mismatch)

Question

Data Drift feature histogram chart shows wrong values against category labels for categorical features (label/value order mismatch)

Ashutosh Singh 0

For a categorical feature monitored under a Data Drift signal (Population Stability Index metric, tabular classification model), I compared the raw histogram JSON output (from model_monitor_metrics_output/signals/data_drift_signal/<feature>.histogram.json) against what Studio's "Feature breakdown" histogram chart displays for the same feature.

The raw JSON lists each category with its baselineCount/targetCount, in an arbitrary (not alphabetically sorted) order, e.g.:

category A: baselineCount 144102, targetCount 2648

category B: baselineCount 57231, targetCount 1016

category C: baselineCount 34449, targetCount 484

category D: baselineCount 181921, targetCount 10571

...

In Studio's histogram chart for this feature, the x-axis category labels are displayed in ALPHABETICAL order, but the bar heights (values) appear to remain in the ORIGINAL JSON order rather than being re-sorted to match. This causes a mismatch: e.g., hovering over the first (alphabetically first) category's bar shows a "Reference" value that actually belongs to a completely different category in the raw JSON.

I confirmed this by manually re-sorting the JSON array alphabetically by category name (keeping each category's name+value pair intact) — once pre-sorted this way, the chart's labels and values lined up correctly. This suggests the chart component sorts the label axis alphabetically for display but doesn't apply the same reordering to the corresponding value array, so values only end up correctly aligned when the source JSON coincidentally happens to already be in alphabetical order.

This is a significant issue for interpreting drift results, since it silently reassigns real values to the wrong category names without any visual indication that something is wrong — someone relying on the dashboard without cross-checking the raw JSON blob would draw incorrect conclusions about which category is actually drifting or has the largest reference/production count.

Could you confirm whether this is a known bug in the histogram rendering component, and whether there's a fix planned or a workaround (e.g., a way to force the chart to use the JSON's original category order instead of re-sorting alphabetically)?

Thanmayi Godithi 11,565 Reputation points Microsoft External Staff Moderator

2026-08-12T09:49:10.54+00:00

Hi Ashutosh Singh ,

Thank you for the detailed investigation and for including the comparison between the raw histogram JSON and the Studio visualization.

At present, we were unable to find any public documentation describing the ordering contract for the histogram artifact (model_monitor_metrics_output/signals/data_drift_signal/<feature>.histogram.json) or the ordering behavior used by the Studio "Feature breakdown" chart. We also could not locate any published known issue covering this specific scenario.

Based on your observations, it appears that the chart labels and the underlying count values may not be aligned when categorical values are displayed in a different order than the source histogram data. However, because this behavior is not documented, we cannot currently confirm whether this is an intended design choice or a rendering defect.

To help investigate further, could you share:

A screenshot of the Feature breakdown chart showing the affected category values.

A sanitized sample of the corresponding <feature>.histogram.json output.

The Azure ML Studio region and workspace type being used.

Whether the issue is reproducible across multiple categorical features and monitoring runs.

If the raw JSON counts consistently differ from the values displayed for the same category in Studio, I recommend opening an Azure Support request and attaching the above evidence. That will allow the engineering team to validate the rendering behavior and determine whether a product fix is required.

As a temporary validation approach, the raw histogram artifact should be treated as the authoritative source when verifying category counts until the discrepancy has been reviewed.

Thanks again for reporting this and providing a detailed reproduction path.
Ashutosh Singh 0 Reputation points

2026-08-13T14:08:59.4466667+00:00

Duplicate comment
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Ashutosh Singh 0 Reputation points

2026-08-13T14:27:20.21+00:00

Hi Thanmayi,

Thank you for the detailed response. I have run several monitoring jobs and observed this same behavior consistently, so I wanted to share what I found.

This appears to be a UI rendering issue in Azure ML Studio where the chart labels are sorted alphabetically, but the metric values remain mapped by their original array index rather than being re-sorted along with the labels.

Here is the reproduction detail from division.histogram.json. The raw JSON lists categories in this order: FHI with baseline count 144102, PBC with 57231, CTA with 34449, MAC with 181921, RTW with 43179, and SWR with 88606.

In the Feature Breakdown chart, the x-axis re-orders these same categories alphabetically as CTA, FHI, MAC, PBC, RTW, SWR. However, the bar heights do not move with the labels. They stay in their original index position. So the bar shown under PBC (the 4th alphabetical position) actually displays 181921, which is the value that belongs to MAC in the raw JSON.

This is reproducible across all categorical features and across every monitoring run I have tested. Azure Region and Workspace: (insert your region and workspace type here).

Since the array ordering in the histogram JSON and the label ordering shown in Studio are out of sync, this causes incorrect values to be displayed against the wrong categories for every categorical feature in Model Monitoring.

I have two screenshots documenting this side by side (JSON artifact and Feature Breakdown chart).

Best regards,

Ashutosh Singh
Thanmayi Godithi 11,565 Reputation points Microsoft External Staff Moderator

2026-08-13T14:50:04.1366667+00:00

Hi Ashutosh Singh ,

Thank you for the detailed analysis. Iam checking this with the Product Group team. I will keep you posted with the updates.
Sina Salam 31,296 Reputation points Volunteer Moderator

2026-08-15T02:28:55.8133333+00:00
Hello Ashutosh Singh,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that your Data Drift feature histogram chart shows wrong values against category labels for categorical features (label/value order mismatch).

We sincerely apologize to hear that the issue has not yet been resolved, and this is not caused by your misconfiguration. Your satisfaction is important to us, and we are committed to working with you toward a resolution.

Based on the evidence provided, the most likely failing layer is the Azure ML Studio visualization path for categorical histograms. However, the exact implementation defect remains unconfirmed until Microsoft engineering reviews the reproduction data. Until Microsoft engineering validates the behavior, I recommend treating the histogram artifact as the authoritative source for categorical distributions and drift analysis.

The issues is under investigation and the category/value pair shown in the Feature Breakdown chart will exactly matches the category/value pair stored in the corresponding .histogram.json artifact for the same monitoring run.

For more insight:

Model monitoring in production - Azure Machine Learning - https://learn.microsoft.com/azure/machine-learning/concept-model-monitoring

Monitor model performance in production - Azure Machine Learning - https://learn.microsoft.com/azure/machine-learning/how-to-monitor-model-performance

I hope this is helpful. Please! Do not hesitate to let me know if you have any other questions, steps or clarifications.

Please do not close the thread by upvoting and accepting the answer if any part of it is helpful.

Your answer

Thanmayi Godithi 11,565 Reputation points Microsoft External Staff Moderator

2026-08-12T09:49:10.54+00:00

Hi Ashutosh Singh ,

Thank you for the detailed investigation and for including the comparison between the raw histogram JSON and the Studio visualization.

At present, we were unable to find any public documentation describing the ordering contract for the histogram artifact (model_monitor_metrics_output/signals/data_drift_signal/<feature>.histogram.json) or the ordering behavior used by the Studio "Feature breakdown" chart. We also could not locate any published known issue covering this specific scenario.

Based on your observations, it appears that the chart labels and the underlying count values may not be aligned when categorical values are displayed in a different order than the source histogram data. However, because this behavior is not documented, we cannot currently confirm whether this is an intended design choice or a rendering defect.

To help investigate further, could you share:

A screenshot of the Feature breakdown chart showing the affected category values.

A sanitized sample of the corresponding <feature>.histogram.json output.

The Azure ML Studio region and workspace type being used.

Whether the issue is reproducible across multiple categorical features and monitoring runs.

If the raw JSON counts consistently differ from the values displayed for the same category in Studio, I recommend opening an Azure Support request and attaching the above evidence. That will allow the engineering team to validate the rendering behavior and determine whether a product fix is required.

As a temporary validation approach, the raw histogram artifact should be treated as the authoritative source when verifying category counts until the discrepancy has been reviewed.

Thanks again for reporting this and providing a detailed reproduction path.
Ashutosh Singh 0 Reputation points

2026-08-13T14:08:59.4466667+00:00

Duplicate comment
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Ashutosh Singh 0 Reputation points

2026-08-13T14:27:20.21+00:00

Hi Thanmayi,

Thank you for the detailed response. I have run several monitoring jobs and observed this same behavior consistently, so I wanted to share what I found.

This appears to be a UI rendering issue in Azure ML Studio where the chart labels are sorted alphabetically, but the metric values remain mapped by their original array index rather than being re-sorted along with the labels.

Here is the reproduction detail from division.histogram.json. The raw JSON lists categories in this order: FHI with baseline count 144102, PBC with 57231, CTA with 34449, MAC with 181921, RTW with 43179, and SWR with 88606.

In the Feature Breakdown chart, the x-axis re-orders these same categories alphabetically as CTA, FHI, MAC, PBC, RTW, SWR. However, the bar heights do not move with the labels. They stay in their original index position. So the bar shown under PBC (the 4th alphabetical position) actually displays 181921, which is the value that belongs to MAC in the raw JSON.

This is reproducible across all categorical features and across every monitoring run I have tested. Azure Region and Workspace: (insert your region and workspace type here).

Since the array ordering in the histogram JSON and the label ordering shown in Studio are out of sync, this causes incorrect values to be displayed against the wrong categories for every categorical feature in Model Monitoring.

I have two screenshots documenting this side by side (JSON artifact and Feature Breakdown chart).

Best regards,

Ashutosh Singh
Thanmayi Godithi 11,565 Reputation points Microsoft External Staff Moderator

2026-08-13T14:50:04.1366667+00:00

Hi Ashutosh Singh ,

Thank you for the detailed analysis. Iam checking this with the Product Group team. I will keep you posted with the updates.
Sina Salam 31,296 Reputation points Volunteer Moderator

2026-08-15T02:28:55.8133333+00:00

Hello Ashutosh Singh,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that your Data Drift feature histogram chart shows wrong values against category labels for categorical features (label/value order mismatch).

We sincerely apologize to hear that the issue has not yet been resolved, and this is not caused by your misconfiguration. Your satisfaction is important to us, and we are committed to working with you toward a resolution.

Based on the evidence provided, the most likely failing layer is the Azure ML Studio visualization path for categorical histograms. However, the exact implementation defect remains unconfirmed until Microsoft engineering reviews the reproduction data. Until Microsoft engineering validates the behavior, I recommend treating the histogram artifact as the authoritative source for categorical distributions and drift analysis.

The issues is under investigation and the category/value pair shown in the Feature Breakdown chart will exactly matches the category/value pair stored in the corresponding .histogram.json artifact for the same monitoring run.

For more insight:

Model monitoring in production - Azure Machine Learning - https://learn.microsoft.com/azure/machine-learning/concept-model-monitoring

Monitor model performance in production - Azure Machine Learning - https://learn.microsoft.com/azure/machine-learning/how-to-monitor-model-performance

I hope this is helpful. Please! Do not hesitate to let me know if you have any other questions, steps or clarifications.

Please do not close the thread by upvoting and accepting the answer if any part of it is helpful.