Azure Storage Blob to ADX daily cost data ingestion not updating

Question

Azure Storage Blob to ADX daily cost data ingestion not updating

B, Hema 0

Hi Team,

I am working on an Azure Cost Management dashboard using the following architecture:

Azure Cost Export->StorageAccount(Blob)-> Azure Data Explorer(ADX)-> Grafana

Current status :

Cost files are getting generated daily in Azure Storage Blob successfully.
ADX table and schema are created
Grafana is connected to ADX as a datasource

Problem:

The daily cost files from Blob are NOT getting ingested/updated into ADX automatically.

Because of this, my ADX table is not receiving new record and Grafana dashboards show "No data".

Expected behavior :

Daily cost export files should automatically ingest into ADX and reflect in Grafana dashboards

What I tried :

Verified storage files exits
Checked ADX ingestion mapping
Grafana works only old/manual data

Questions :

How can I automatically ingest daily Blob files into ADX?
What is recommended approach for daily cost export ingestion?

Please guide me on the correct setup. So, that i can update my work

Thanks

Bharath Y P 5,620 Reputation points Microsoft External Staff Moderator

2026-02-09T20:08:52.03+00:00
Hello B, Hema, It sounds like you're facing quite a challenge with your Azure Cost Management dashboard. Since your daily cost files from the Azure Storage Blob aren’t updating in Azure Data Explorer (ADX), let's go over a few things you can try and the recommended approach for automatic ingestion.

Steps to Resolve Your Issue:

Configure Continuous Ingestion: Make sure you set up continuous ingestion from your Azure Blob Storage to ADX. This can be done using either an Event Grid trigger or a scheduled job that picks up the new blob files. You can refer to the Event Grid data connection for more details.

Use an Ingestion Mapping: Verify that the ingestion mapping for the ADX table you created matches the structure of the JSON or CSV files in Blob Storage. Mismatches can prevent successful ingestion. Check your ingestion mapping settings again.

Ingestion Validation: Utilize the diagnostics and logs available in ADX to check for any errors related to ingestion. You can explore the Azure Data Explorer Diagnostics feature to help identify issues.

Check Storage Permissions: Ensure that ADX has the necessary permissions to access the Blob Storage account. You might need to configure the correct role assignments in Azure IAM.

Run a Manual Ingestion: To confirm that the ingestion setup is working, try manually ingesting a blob file into ADX using the Azure portal or command-line interface to see if it succeeds.

Recommended Approach for Daily Cost Export Ingestion:

Event Grid Trigger: Setting up an Event Grid trigger that detects when new blobs are created in your storage account and automatically starts the ingestion process into ADX is a highly recommended approach. This allows real-time updates without manual intervention.

Scheduled Ingestion Job: If real-time ingestion isn't feasible, you can schedule a job using Azure Data Factory or Azure Logic Apps to run daily and check for new blobs to ingest.

By implementing one of these methods, your ADX table should reflect the new cost data daily, allowing your Grafana dashboards to update accordingly.

Reference Documentation:

Event Grid Data Connection

Managing Permissions in Azure

I hope this helps! If you have any more questions or need further assistance, feel free to ask. Thanks

1 answer

Your answer

Answer 1

There could be several reasons why you are unable to send your Azure cost data daily updates from Azure Blob Storage to Azure Data Explorer (ADX). Here are some common issues to consider:

Configuration Issues: Ensure that the data pipeline or integration setup between Azure Blob Storage and ADX is correctly configured. Check the settings for the data ingestion process to ensure they match the required parameters.
Data Format: Verify that the data format in Azure Blob Storage is compatible with ADX. If the data is not in the expected format, it may fail to ingest.
Access Permissions: Confirm that the necessary permissions are granted for both Azure Blob Storage and ADX. The service principal or identity used for the ingestion must have the appropriate access rights.
Error Handling: Check for any error messages or logs that might indicate what the issue is. Azure provides error codes and messages that can help diagnose problems with data ingestion.
Ingestion Limits: Ensure that you are not exceeding any ingestion limits set by ADX, which could prevent new data from being ingested.

If you continue to experience issues, consider reviewing the Azure documentation or reaching out to Azure support for further assistance.

References:

Share via

Azure Storage Blob to ADX daily cost data ingestion not updating

1 answer

Your answer