How to Upload Carbon Black Logs and Alerts into Azure Sentinel for Evaluation

Question

I am trying to evaluate how much Azure Sentinel helps my business's security needs. I am particularly interested in seeing how well Azure Sentinel can cluster alerts together. I have taken a small amount of EDR logs and alerts (which are in json format) internally within my company and am now trying to figure out how to upload those logs into Azure Sentinel so they can be processed. However, I do not see any directions on how to do this online. I have two files uploaded to an Azure Storage Account; one contains all the EDR logs, and the other contains all the alerts. The Microsoft Sales Rep told me it was possible to test Sentinel against the data I had collected. I do not want to connect Sentinel directly to my CB account.

I was told to follow these directions by the rep to do this but I am unable to figure out how to take data from the Azure Storage account into Sentine ie: how to link storage account to Sentinel:

1. Prepare Your Data: Ensure your sample data is in a supported format, such as CSV, JSON, or TXT.

2.Create an Azure Storage Account:

Go to the Azure portal and create a new storage account.

Upload your sample data files to a container in this storage account.

3. Set Up Microsoft Sentinel:

In the Azure portal, navigate to Microsoft Sentinel and create a new workspace if you don't have one.

Link your storage account to Sentinel.

4. Ingest Data:

Use the "Custom Logs" feature in Sentinel to define the structure of your data.

Configure the data ingestion settings to pull data from your storage account.

5. Analyze Your Data:

Once the data is ingested, you can use Sentinel’s built-in analytics and query capabilities to analyze your sample data.

Answer 1

I think there is a misunderstanding here on how data is ingested into Sentinel. It is very uncommon and difficult to ingest historic data in the manner you are describing. Rather than trying to pull in historic files from blob storage, the best way is to send active logs using one of the provided data connectors. They will ingest data directly from Microsoft solutions, from a monitoring agent (Microsoft or Logstash), from a log forwarder (Syslog/CEF), or from an API [depending on the specific data connector. Your CB logs will likely use Syslog over the forwarder or an API based connection.

If you are interested in alert correlation, Sentinel has analytic rules for generating alerts. Either from customizable templates included with each connector or custom rules. Alerts are not automatically correlated within Sentinel by default. These rules do have some limited alert grouping capabilities if configured. With the new Unified XDR portal, Sentinel now sends all alerts to the Unified XDR portal which automatically correlates all Microsoft and 3rd party alerts into incidents.

In short, ingest your data into Sentinel using a connector, use analytic rules to generate alerts, configure alert grouping at the rule level if needed, though in most cases the Unified XDR portal will perform alert correlation automatically.

Answer 2

@psec-comp Thank you for reaching out to us. As I understand that you are interested in assessing how well Microsoft Sentinel can cluster alerts together and you are looking to upload a small amount of EDR logs and alerts in JSON format to Azure Sentinel for processing.

I would like to suggest that you refer to the following resources that explain how to ingest logs of any format in Azure Sentinel:

https://www.youtube.com/watch?v=Voewqmt8xr0&list=PL8wOlV8Hv3o8ri_K_8c2THT_4ZJ_KKl90&index=12

https://www.youtube.com/watch?v=fzHyOqLPxCY&list=PL8wOlV8Hv3o8ri_K_8c2THT_4ZJ_KKl90&index=14

These videos provide step-by-step instructions on how to ingest logs of any format in Azure Sentinel. While they do not specifically cover how to use an Azure Storage account to upload logs, they should provide you with the information you need to get started.

I hope this helps. If you have any further questions or concerns, please let me know.

Reference: https://techcommunity.microsoft.com/blog/microsoftsentinelblog/microsoft-sentinel-support-for-ingestion-time-data-transformations/3244531

Please remember to "Accept Answer" if answer helped, so that others in the community facing similar issues can easily find the solution.