How does Azure Data Explorer compare to Log Analytics and Azure Data Factory for log analysis with KQL and Power BI integration?

Question

How does Azure Data Explorer compare to Log Analytics and Azure Data Factory for log analysis with KQL and Power BI integration?

NK, SatishKumar 81

"I'm currently using Log Analytics and Azure Data Factory to capture, transform, and analyze logs for my application on Azure. However, this solution is becoming too costly for my budget. I'm considering using Azure Data Explorer as a single solution for my log analysis needs. I'm currently using ADF just for lookup and adding additional attributes, and then storing the transformed logs back to Log Analytics Workspace as a custom table. Finally, I'm running KQL queries to power BI reports. Has anyone else used ADX for this purpose? How does it compare to Log Analytics and ADF in terms of cost, scalability, and ease of use? Are there any limitations or challenges that I should be aware of before making the switch?"

Andrew Blumhardt 10,051 Reputation points Microsoft Employee

2023-05-30T04:38:23.56+00:00

I think it is definitely worth the time to research and experiment with ADX. I know it supports ADF and PowerBI. I can't say for certain of it will be less expensive but likely yes. It is certainly more complex to setup and maintain. You might find some new options that you like over workspaces as well.

1 answer

Your answer

Andrew Blumhardt 10,051 Reputation points Microsoft Employee

2023-05-30T04:38:23.56+00:00

I think it is definitely worth the time to research and experiment with ADX. I know it supports ADF and PowerBI. I can't say for certain of it will be less expensive but likely yes. It is certainly more complex to setup and maintain. You might find some new options that you like over workspaces as well.

Answer 1

David Broggy 6,371 MVP Volunteer Moderator

Hi Satish,

Once you're ok with the cost of the ADX instance the cost is quite reasonable, since you can use blob storage for your data.

So I'd suggest calculating your monthly cost for an adx instance (eg. .095x24hoursx30days=$68US?)

And then add on top of that your storage needs and you'll have a close estimate.

If speed isn't critical (and it shouldn't be for scheduled reporting) consider using blob storage since you can still query your data using kql.

If your reports contain a very large amount of data you'll want to do some performance testing to ensure the query times aren't an issue.

reference:

https://azure.microsoft.com/en-us/pricing/details/data-explorer/

NK, SatishKumar 81 Reputation points

2023-05-29T14:05:24.2833333+00:00

Thanks @David Broggy , here i am also looking for re solutioning "I'm considering ingesting all my logs to Azure Data Explorer and connecting to Cosmos DB for my master data. If I do this, can I eliminate the need for Azure Data Factory for transformation? Can I write a KQL query to join data from both Azure Data Explorer and Cosmos DB?

Share via

How does Azure Data Explorer compare to Log Analytics and Azure Data Factory for log analysis with KQL and Power BI integration?

1 answer

Your answer