Export Logs from Log Analytics Workspace to Blob Storage

Question

Export Logs from Log Analytics Workspace to Blob Storage

Adriaan Boshoff 5

Hi all,

I have a Log Analytics Workspace that is linked to Sentinel.

I have a lot of logs that I need to export from the Workspace into Blob Storage.

Th logs date back 30 days and it is about 400GB, it is about 500 million logs.

Please let me know what the best way is to approach this.

Thank you

4 answers

Your answer

Answer 1

EduardsGrebezs 1,186

Hey,

Just one question. You need to export logs to store them related to compliance? If so i think the cheapest way is to use New Azure Data Lake feature.

Otherwise Logic app could be used. You could check this one - https://techcommunity.microsoft.com/blog/microsoftsentinelblog/move-your-microsoft-sentinel-logs-to-long-term-storage-with-ease/1407153

And this is a solution probably - https://learn.microsoft.com/en-us/azure/azure-monitor/logs/logs-export-logic-app

Answer 2

KarishmaTiwari-MSFT 20,852 Microsoft Employee Moderator

@Adriaan Boshoff

You can also set up data export from your Log Analytics workspace to Azure Data Lake Storage (ADLS Gen2). Azure Synapse Analytics provides tools to query the exported data, including T-SQL scripts, Spark code, and machine learning models. You can analyze the data using Spark for machine learning or complex analytics or use the serverless database for auditing scenarios2.

https://techcommunity.microsoft.com/t5/azure-observability-blog/how-to-analyze-data-exported-from-log-analytics-data-using/ba-p/2547888

Adriaan Boshoff 5 Reputation points

2024-07-12T10:40:36.3+00:00

Thank you Karishma, this seems to only work for live data feeding into the workspace we need to do this for historical data going back 30 days. Or is there a way that you know of to do this for historical data?
KarishmaTiwari-MSFT 20,852 Reputation points Microsoft Employee Moderator

2024-07-19T00:05:22.0066667+00:00

Let me look into it and get back.
Praveen Ambati 1 Reputation point

2025-08-29T07:20:39.98+00:00

@Anonymous

We are facing a similar challenge — we need to transfer terabytes of data from Log Analytics Workspace (LAW) to an Azure Storage Account. We are currently following the guidance here: https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/Sentinel%20Bulk%20Logs%20Export.ipynb .

However, this approach first downloads the data to a local disk before copying it into the Storage Account. This is not ideal, as it requires a very large local disk (since we are dealing with TBs of data), which makes the process expensive and inefficient.

@KarishmaTiwari-MSFT is there a more straightforward way to transfer data directly from LAW to a Storage Account, or perhaps a high-performance service that could fit this scenario?

Answer 3

Carlos Solís Salazar 18,376 MVP Volunteer Moderator

Hi,
You can achieve your goal through Data Export, please follow this documentation

I hope this helps!

Remember to accept the answer if it is helpful.

Adriaan Boshoff 5 Reputation points

2024-07-12T10:38:02.1466667+00:00

Thank you Carlos, this seems to only work for live data feeding into the workspace we need to do this for historical data going back 30 days. Or is there a way that you know of to do this for historical data?

Answer 4

Abiola Akinbade 30,485 Volunteer Moderator

Hello Adriaan Boshoff,

Thanks for your question.

You can leverage logic apps. See how to do so here:

Export data from a Log Analytics workspace to a storage account by using Logic Apps

You can mark it 'Accept Answer' and 'Upvote' if this helped you

Regards,

Abiola

Adriaan Boshoff 5 Reputation points

2024-07-12T10:36:57.3933333+00:00

Thank you, this works, but we are unable to export 500 million logs we have to do it 60000 logs at a time and this logic app ran for 9 hours, so this solution will take a very long time. Do you have a workaround for this? Only 60000 logs are being exported because I believe on queries there is a limit of 500 000 rows or a limit of 64MB per file.

Share via

Export Logs from Log Analytics Workspace to Blob Storage

4 answers

Your answer