Edit

Create and manage custom detections rules

  • Article
  • 11 minutes to read

Note

Want to experience Microsoft 365 Defender? Learn more about how you can evaluate and pilot Microsoft 365 Defender.

Applies to:

  • Microsoft 365 Defender

Custom detection rules are rules you can design and tweak using advanced hunting queries. These rules let you proactively monitor various events and system states, including suspected breach activity and misconfigured endpoints. You can set them to run at regular intervals, generating alerts and taking response actions whenever there are matches.

Required permissions for managing custom detections

To manage custom detections, you need to be assigned one of these roles:

  • Security settings (manage)—Users with this Microsoft 365 Defender permission can manage security settings in the Microsoft 365 Defender portal.

  • Security administrator—Users with this Azure Active Directory role can manage security settings in the Microsoft 365 Defender portal and other portals and services.

  • Security operator—Users with this Azure Active Directory role can manage alerts and have global read-only access to security-related features, including all information in the Microsoft 365 Defender portal. This role is sufficient for managing custom detections only if role-based access control (RBAC) is turned off in Microsoft Defender for Endpoint. If you have RBAC configured, you also need the manage security settings permission for Defender for Endpoint.

You can also manage custom detections that apply to data from specific Microsoft 365 Defender solutions if you have permissions for them. If you only have manage permissions for Microsoft 365 Defender for Office, for instance, you can create custom detections using Email tables but not Identity tables.

To manage required permissions, a global administrator can:

  • Assign the security administrator or security operator role in Microsoft 365 admin center under Roles > Security admin.
  • Check RBAC settings for Microsoft Defender for Endpoint in Microsoft 365 Defender under Settings > Permissions > Roles. Select the corresponding role to assign the manage security settings permission.

Note

To manage custom detections, security operators will need the manage security settings permission in Microsoft Defender for Endpoint if RBAC is turned on.

Create a custom detection rule

1. Prepare the query

In the Microsoft 365 Defender portal, go to Advanced hunting and select an existing query or create a new query. When using a new query, run the query to identify errors and understand possible results.

Important

To prevent the service from returning too many alerts, each rule is limited to generating only 100 alerts whenever it runs. Before creating a rule, tweak your query to avoid alerting for normal, day-to-day activity.

Required columns in the query results

To create a custom detection rule, the query must return the following columns:

  • Timestamp—used to set the timestamp for generated alerts
  • ReportId—enables lookups for the original records
  • One of the following columns that identify specific devices, users, or mailboxes:
    • DeviceId
    • DeviceName
    • RemoteDeviceName
    • RecipientEmailAddress
    • SenderFromAddress (envelope sender or Return-Path address)
    • SenderMailFromAddress (sender address displayed by email client)
    • RecipientObjectId
    • AccountObjectId
    • AccountSid
    • AccountUpn
    • InitiatingProcessAccountSid
    • InitiatingProcessAccountUpn
    • InitiatingProcessAccountObjectId

Note

Support for additional entities will be added as new tables are added to the advanced hunting schema.

Simple queries, such as those that don't use the project or summarize operator to customize or aggregate results, typically return these common columns.

There are various ways to ensure more complex queries return these columns. For example, if you prefer to aggregate and count by entity under a column such as DeviceId, you can still return Timestamp and ReportId by getting it from the most recent event involving each unique DeviceId.

Important

Avoid filtering custom detections using the Timestamp column. The data used for custom detections is pre-filtered based on the detection frequency.

The sample query below counts the number of unique devices (DeviceId) with antivirus detections and uses this count to find only the devices with more than five detections. To return the latest Timestamp and the corresponding ReportId, it uses the summarize operator with the arg_max function.

DeviceEvents
| where ingestion_time() > ago(1d)
| where ActionType == "AntivirusDetection"
| summarize (Timestamp, ReportId)=arg_max(Timestamp, ReportId), count() by DeviceId
| where count_ > 5

Tip

For better query performance, set a time filter that matches your intended run frequency for the rule. Since the least frequent run is every 24 hours, filtering for the past day will cover all new data.

2. Create new rule and provide alert details

With the query in the query editor, select Create detection rule and specify the following alert details:

  • Detection name—name of the detection rule; should be unique
  • Frequency—interval for running the query and taking action. See additional guidance below
  • Alert title—title displayed with alerts triggered by the rule; should be unique
  • Severity—potential risk of the component or activity identified by the rule
  • Category—threat component or activity identified by the rule
  • MITRE ATT&CK techniques—one or more attack techniques identified by the rule as documented in the MITRE ATT&CK framework. This section is hidden for certain alert categories, including malware, ransomware, suspicious activity, and unwanted software
  • Description—more information about the component or activity identified by the rule
  • Recommended actions—additional actions that responders might take in response to an alert

Rule frequency

When you save a new rule, it runs and checks for matches from the past 30 days of data. The rule then runs again at fixed intervals, applying a lookback duration based on the frequency you choose:

  • Every 24 hours—runs every 24 hours, checking data from the past 30 days
  • Every 12 hours—runs every 12 hours, checking data from the past 48 hours
  • Every 3 hours—runs every 3 hours, checking data from the past 12 hours
  • Every hour—runs hourly, checking data from the past 4 hours
  • Continuous (NRT)—runs continuously, checking data from events as they are collected and processed in near real-time

Note

If you choose the continuous frequency, make sure that the query references one table only and uses an operator from the list of supported KQL operators. You cannot use unions or joins. The externaldata operator is not supported.

When you edit a rule, it will run with the applied changes in the next run time scheduled according to the frequency you set. The rule frequency is based on the event timestamp and not the ingestion time.

Tip

Match the time filters in your query with the lookback duration. Results outside of the lookback duration are ignored.

Select the frequency that matches how closely you want to monitor detections. Consider your organization's capacity to respond to the alerts.

Tables that support Continuous (NRT) frequency

Near real-time detections are supported for the following tables:

  • AlertEvidence
  • DeviceEvents
  • DeviceFileCertificateInfo
  • DeviceFileEvents
  • DeviceImageLoadEvents
  • DeviceLogonEvents
  • DeviceNetworkEvents
  • DeviceNetworkInfo
  • DeviceInfo
  • DeviceProcessEvents
  • DeviceRegistryEvents
  • EmailAttachmentInfo
  • EmailEvents
  • EmailPostDeliveryEvents
  • EmailUrlInfo
  • UrlClickEvents

Note

Only columns that are generally available can support Continuous (NRT) frequency.

3. Choose the impacted entities

Identify the columns in your query results where you expect to find the main affected or impacted entity. For example, a query might return sender (SenderFromAddress or SenderMailFromAddress) and recipient (RecipientEmailAddress) addresses. Identifying which of these columns represent the main impacted entity helps the service aggregate relevant alerts, correlate incidents, and target response actions.

You can select only one column for each entity type (mailbox, user, or device). Columns that are not returned by your query can't be selected.

4. Specify actions

Your custom detection rule can automatically take actions on devices, files, users, or emails that are returned by the query.

Screenshot that shows actions for custom detections in the Microsoft 365 Defender portal.

Actions on devices

These actions are applied to devices in the DeviceId column of the query results:

Actions on files

  • When selected, the Allow/Block action can be applied to the file. Blocking files are only allowed if you have Remediate permissions for files and if the query results have identified a file ID, such as a SHA1. Once a file is blocked, other instances of the same file in all devices are also blocked. You can control which device group the blocking is applied to, but not specific devices.

  • When selected, the Quarantine file action can be applied to files in the SHA1, InitiatingProcessSHA1, SHA256, or InitiatingProcessSHA256 column of the query results. This action deletes the file from its current location and places a copy in quarantine.

Actions on users

  • When selected, the Mark user as compromised action is taken on users in the AccountObjectId, InitiatingProcessAccountObjectId, or RecipientObjectId column of the query results. This action sets the users risk level to "high" in Azure Active Directory, triggering corresponding identity protection policies.

  • Select Disable user to temporarily prevent a user from logging in.

  • Select Force password reset to prompt the user to change their password on the next sign in session.

Both the Disable user and Force password reset options require the user SID, which are in the columns AccountSid, InitiatingProcessAccountSid, RequestAccountSid, and OnPremSid.

For more details on user actions, read Remediation actions in Microsoft Defender for Identity.

Actions on emails

  • If the custom detection yields email messages, you can select Move to mailbox folder to move the email to a selected folder (any of Junk, Inbox, or Deleted items folders).

  • Alternatively, you can select Delete email and then choose to either move the emails to Deleted Items (Soft delete) or delete the selected emails permanently (Hard delete).

The columns NetworkMessageId and RecipientEmailAddress must be present in the query output to apply actions to email messages.

5. Set the rule scope

Set the scope to specify which devices are covered by the rule. The scope influences rules that check devices and doesn't affect rules that check only mailboxes and user accounts or identities.

When setting the scope, you can select:

  • All devices
  • Specific device groups

Only data from devices in scope will be queried. Also, actions will be taken only on those devices.

6. Review and turn on the rule

After reviewing the rule, select Create to save it. The custom detection rule immediately runs. It runs again based on configured frequency to check for matches, generate alerts, and take response actions.

Important

Custom detections should be regularly reviewed for efficiency and effectiveness. To make sure you are creating detections that trigger true alerts, take time to review your existing custom detections by following the steps in Manage existing custom detection rules.

You maintain control over the broadness or specificity of your custom detections so any false alerts generated by custom detections might indicate a need to modify certain parameters of the rules.

Manage existing custom detection rules

You can view the list of existing custom detection rules, check their previous runs, and review the alerts they have triggered. You can also run a rule on demand and modify it.

Tip

Alerts raised by custom detections are available over alerts and incident APIs. For more information, see Supported Microsoft 365 Defender APIs.

View existing rules

To view all existing custom detection rules, navigate to Hunting > Custom detection rules. The page lists all the rules with the following run information:

  • Last run—when a rule was last run to check for query matches and generate alerts
  • Last run status—whether a rule ran successfully
  • Next run—the next scheduled run
  • Status—whether a rule has been turned on or off

View rule details, modify rule, and run rule

To view comprehensive information about a custom detection rule, go to Hunting > Custom detection rules and then select the name of rule. You can then view general information about the rule, including information, its run status, and scope. The page also provides the list of triggered alerts and actions.

The Custom detection rule details page in the Microsoft 365 Defender portal

You can also take the following actions on the rule from this page:

  • Run—run the rule immediately. This also resets the interval for the next run.
  • Edit—modify the rule without changing the query
  • Modify query—edit the query in advanced hunting
  • Turn on / Turn off—enable the rule or stop it from running
  • Delete—turn off the rule and remove it

View and manage triggered alerts

In the rule details screen (Hunting > Custom detections > [Rule name]), go to Triggered alerts, which lists the alerts generated by matches to the rule. Select an alert to view detailed information about it and take the following actions:

  • Manage the alert by setting its status and classification (true or false alert)
  • Link the alert to an incident
  • Run the query that triggered the alert on advanced hunting

Review actions

In the rule details screen (Hunting > Custom detections > [Rule name]), go to Triggered actions, which lists the actions taken based on matches to the rule.

Tip

To quickly view information and take action on an item in a table, use the selection column [✓] at the left of the table.

Note

Some columns in this article might not be available in Microsoft Defender for Endpoint. Turn on Microsoft 365 Defender to hunt for threats using more data sources. You can move your advanced hunting workflows from Microsoft Defender for Endpoint to Microsoft 365 Defender by following the steps in Migrate advanced hunting queries from Microsoft Defender for Endpoint.

See also