Tune alerts from migrated custom detections in Microsoft Defender

Create a temporary alert tuning rule to auto-resolve evaluation alerts while you validate migrated custom detections. Migrated custom detections use #EVAL in their alert titles during evaluation. Match both the detection source and the title so that the rule doesn't resolve unrelated alerts.

Before you begin, review the access and Preview availability prerequisites.

Prerequisites

  • Access to Settings > Microsoft Defender XDR > Alert tuning in the Microsoft Defender portal.
  • Custom detection available in the Detection source list. Detection source selection is a Preview capability, and availability can differ between tenants. Create the rule only after this option appears.
  • Migrated custom detections under evaluation, with #EVAL in their alert titles.

Create the temporary tuning rule

Use both matching conditions to restrict the rule to migrated custom detection evaluation alerts.

Important

Alert tuning can change alert visibility, incidents, notifications, and automation. Keep both conditions: Detection source = Custom detection and Alert title contains #EVAL. Don't use the title condition alone.

  1. Sign in to the Microsoft Defender portal.

  2. Go to Settings > Microsoft Defender XDR > Alert tuning.

  3. Select Create alert tuning rule.

  4. Configure the following values:

    Setting Value
    Rule name Suppress migrated custom detection evaluation alerts
    Detection source Custom detection
    Condition Alert title contains #EVAL
    Action Auto-resolve
    Status Enabled only for the migration validation period
  5. Review the scope, create the rule, and confirm that it's enabled.

Validate matching and incident behavior

Matching alerts should be auto-resolved. Each is expected to create a resolved one-alert, or singleton, incident. Closed incidents aren't reopened or merged, so these evaluation alerts shouldn't correlate between migrated rules or into existing incidents. Validate this behavior in your tenant.

  1. Positive test: Confirm that a #EVAL custom-detection alert is auto-resolved and that its incident is resolved with that alert as its only alert.
  2. Correlation test: Generate alerts from two different #EVAL custom detections. Confirm that they remain in separate resolved singleton incidents and aren't added to an existing incident.
  3. Title test: Confirm that a custom-detection alert without #EVAL isn't auto-resolved by this rule.
  4. Source test: Confirm that an alert from another detection source isn't auto-resolved by this rule, even if its title contains #EVAL.

Remove the rule after validation

When migration validation is complete, disable or delete the temporary tuning rule. If other migrated detections still use the same rule for evaluation, wait until their validation is complete.