Microsoft Sentinel components and patterns

This article discusses the different components of a Microsoft Sentinel solution and how they can work together to address important customer scenarios.

The Sentinel platform includes a data lake, graph, Jupyter notebook jobs, a Model Context Protocol (MCP) server, and data from more than 300 Sentinel connectors to help customers centralize and analyze their security data in a cost-efficient way. These capabilities plus Microsoft Security Copilot enable customers and partners to create impactful solutions, which can be published through the Microsoft Security Store.

Sentinel SIEM is used by Security Operations (SOC) teams to generate detections, investigate malicious behavior, and remediate threats. By creating Sentinel connectors to bring in new data, and by creating content such as analytics rules, playbooks, hunting queries, parsers, and workbooks, partners can help SOC teams get information they need to identify threats and respond appropriately. Sentinel SIEM solutions are published through Sentinel’s Content Hub.

Data collection

Whether you’re building a solution that uses platform components or targeting a Sentinel SIEM integration, it’s critical to have the right data for your scenario.

Sentinel Connectors bring in data to Sentinel, which can then be analyzed in the lake using Jupyter notebooks and jobs, or addressed with Sentinel SIEM content such as analytics rules and hunting queries.

That data can include the following types:

Type	Description
Unprocessed data	Supports detections and hunting processes. Analyze raw operational data in which signs of malicious activity may be present. Bring unprocessed data to Microsoft Sentinel to use Microsoft Sentinel's built-in hunting and detection features to identify new threats and more. Examples: Syslog data, CEF data over Syslog, application, firewall, authentication, or access logs, and more.
Security conclusions	Creates alert visibility and opportunity for correlation. Alerts and detections are conclusions that have already been made about threats. Putting detections in context with all the activities and other detections visible in Microsoft Sentinel investigations, saves time for analysts and creates a more complete picture of an incident, resulting in better prioritization and better decisions. Examples: anti-malware alerts, suspicious processes, communication with known bad hosts, network traffic that was blocked and why, suspicious logons, detected password spray attacks, identified phishing attacks, data exfiltration events, and more.
Reference data	Builds context with referenced environments, saving investigation effort and increasing efficiency. Examples: CMDBs, high value asset databases, application dependency databases, IP assignment logs, threat intelligence collections for enrichment, and more.
Threat intelligence	Powers threat detection by contributing indicators of known threats. Threat intelligence can include current indicators that represent immediate threats or historical indicators that are kept for future prevention. Historical data sets are often large and are best referenced ad-hoc, in place, instead of importing them directly to Microsoft Sentinel.

Parsers

Parsers are KQL functions which transform custom data from third-party products into a normalized ASIM schema. Normalization ensures that SOC analysts don’t have to learn details about new schemas and instead build analytic rules and hunting queries on the normalized schema that they're already familiar with. Review the available ASIM schemas provided by Microsoft Sentinel to identify relevant ASIM schemas (one or more) for your data to ensure easier onboarding for SOC analysts and to ensure that the existing security content written for the ASIM schema is applicable out-of-the-box for your product data. For more information on the available ASIM schemas, see Advanced Security Information Model (ASIM) schemas.

Visualization

You can include visualizations to help customers manage and understand your data, by including graphical views of how well data flows into Microsoft Sentinel, and how effectively it contributes to detections.

Monitoring and detection

Sentinel’s monitoring and detection features create automated detections to help customers scale their SOC team's expertise.

The following sections describe monitoring and detection elements that you can include in your solution.

Security Copilot agents

Security Copilot agents automate repetitive tasks and reduce manual workloads. They enhance security and IT operations across cloud, data security and privacy, identity, and network security. For Sentinel, agents can query the SIEM or data lake and call APIs to enrich Microsoft Sentinel data. They can utilize notebook jobs for intensive data processing or analysis and utilize any number of plug-ins.

Jupyter notebook jobs

Jupyter notebook jobs provide powerful tools for performing complex data transformations and running machine learning models using Spark jobs in Sentinel Data lake. They can be used by Security Copilot agents to provide a deterministic and efficient means of performing data analysis and summarization and run on an ongoing basis. Notebook jobs can write custom data tables to the analytic tier and data lake to be used by downstream components such as agents, workbooks, hunting queries, and others.

Analytics rules

Analytics rules are sophisticated detections that can create accurate, meaningful alerts.

Add analytics rules to your solution to help your customers benefit from data from your system in Microsoft Sentinel. For example, analytics rules can help provide expertise and insight about the activities that can be detected in the data your integration delivers.

They can output alerts (notable events), incidents (units of investigation), or trigger automation playbooks.

You can add analytics rules by including them in a solution and via the Microsoft Sentinel ThreatHunters community. Contribute via the community to encourage community creativity over partner-sourced data, helping customers with more reliable and effective detections.

Hunting queries

Hunting queries enable SOC analysts to proactively look for new anomalies that aren't detected by the currently scheduled analytics rules. Hunting queries guide SOC analysts into asking the right questions to find issues from the data that is already available in Microsoft Sentinel and helps them identify potential threat scenarios. By including hunting queries, you can help customers find unknown threats in the data you supply.

Workbooks

Workbooks provide interactive reports and dashboards that help users to visualize security data and identify patterns within data. The need for workbooks depends on the specific use case. As you design your solution, think of scenarios which might be best explained visually, particularly for scenarios to track performance.

Investigation

The Sentinel investigation graph provides investigators with relevant data when they need it, providing visibility about security incidents and alerts via connected entities. Investigators can use the investigation graph to find relevant or related, contributing events to the threat that's under investigation.

Partners can contribute to the investigation graph by providing:

Microsoft Sentinel alerts and incidents, created via analytics rules in partner solutions.
Custom exploration queries for partner-supplied data. Custom exploration queries provide rich exploration and connectivity between data and insights for security investigators.

Response

Playbooks support workflows with rich automation, running security-related tasks across customer environments. They’re critical to ensure that the SOC analysts aren't overburdened by tactical items and can focus on the more strategic and deeper root cause of the vulnerabilities. For example, if a high-severity alert is detected, a playbook can automatically initiate a series of actions, such as notifying the security team, isolating affected systems, and gathering relevant logs for further analysis.

For example, playbooks can help in any of the following ways, and more:

Helping customers configure security policies in partner products
Gathering extra data to inform investigative decisions
Linking Microsoft Sentinel incidents to external management systems
Integrating alert lifecycle management across partner solutions

As you design your solution, think of the automated actions that can be taken to resolve incidents created by the analytic rules defined in your solution.

Sentinel SIEM scenario examples

The following sections describe common partner scenarios, and recommendations for what to include in a solution for each scenario.

Your product generates data that is important to security investigations

Scenario: Your product generates data that can inform security investigations.

Example: Products that supply some form of log data include firewalls, cloud application security brokers, physical access systems, Syslog output, commercially available and enterprise-built LOB applications, servers, network metadata, anything deliverable over Syslog in Syslog or CEF format, or over REST API in JSON format.

How to use your data in Microsoft Sentinel: Import your product's data into Microsoft Sentinel via a data connector to provide analytics, hunting, investigations, visualizations, and more.