Get insights from Telemetry data

Rushank 1 Reputation point
2022-11-24T09:15:00.103+00:00

Hi,

I am trying to get all my telemetry data into PowerBI.

1) So i am using application insights to capture telemetry data and then storing this data into azure blob storage ( because application insights only allow for limited time storage).
Then i am using PowerBi azure storage connector to PowerBI.

2) The problem is data is way too big(eg 100 gb of data in last 2 weeks and increasing). PowerBi refresh cant handle this much data. I even tried to reduce the rows and columns applying all the best practices. But the data is too large.

Current data infrastructure

Application insight -> Azure storage -> PowerBI
What tool/method i should use

  1. Direct query
  2. Azure data explorer
  3. Any other tool/method

The final data insight/dashboard should be in PowerBI

Thank you and have a ncie day !

Regards,
Rushank Patil

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
484 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
331 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Maxim Sergeev 6,566 Reputation points Microsoft Employee
    2022-11-25T04:40:16.817+00:00

    Hi there,

    Application Insights integrated with Log Analytics could store data up to 730 days. In most cases it's more than enough, no?
    https://learn.microsoft.com/en-us/azure/azure-monitor/app/data-retention-privacy#how-long-is-the-data-kept

    It's not a cheapest solution but most easily to maintain and configure for PowerBI

    0 comments