Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Important
Lakewatch is in Private Preview. To request access, reach out to your Azure Databricks account team.
A signal is an event record produced when one or more events match the criteria of a detection rule, such as multiple failed login attempts from a specific user. Each signal includes a summary of the detected behavior, when the first event occurred, and the number of matching events. Each signal attaches to exactly one alert.
To view signals, go to Alerts and open an alert to see its related signals. To query signals directly, use the signals system table.
Signal deduplication
Lakewatch automatically prevents duplicate signals from cluttering your workspace. Two signals are duplicates when a rule produces a result row whose values are identical across all detection output fields (not only the summary) to a signal that the same rule already created within the deduplication time window. Deduplication is scoped to a single rule, so different rules never deduplicate against each other.
When Lakewatch detects a duplicate, it skips that signal entirely: no new signal is inserted and no alert is opened or updated.
Deduplication is distinct from signal grouping, which happens later. A matching summary only leads to deduplication when the rest of the output row matches, too. A matching summary alone leads to grouping.
- Deduplication runs when signals are created and drops a signal only when it matches an existing signal across every detection-output field. Two signals that share a summary but differ in any other fields are both created.
- Signal grouping then organizes the surviving signals into alerts using the summary (plus rule ID and version), and nothing else. Signals that share the opening signal's summary attach to the same alert, even when their other fields differ.
Because a detection rule keeps no checkpoint of the rows it has already processed, a delayed run, a backfill, or a re-run can scan an overlapping time window and return the same rows multiple times. Deduplication ensures those overlaps don't generate duplicate signals.
Enable or disable deduplication
Deduplication is enabled by default for every rule. You can control it at two levels:
- Workspace: Sets the default for all rules. Toggle Signal deduplication in your workspace alert settings. Enabled by default.
- Per rule: Overrides the workspace default for a single rule, using the rule's Deduplication setting:
- Default: Inherit the workspace setting.
- Enabled: Always deduplicate this rule's signals.
- Disabled: Never deduplicate this rule's signals.
For where to set the per-rule values, see Create and manage detection rules.
Deduplication time window
The time window controls how far back Lakewatch looks for an identical signal. By default, the window is automatic: 3× the rule's schedule interval, capped at 7 days. For example, a rule that runs hourly uses a 3-hour window, and a rule that runs daily uses a 3-day window.
To set the window yourself, specify an explicit duration such as 1h, 24h, or 7d (7 days is the maximum). Databricks recommends leaving the window on automatic.
Signals system table
Lakewatch provides a signals system table at <LW-CATALOG>.internal.signals in each Lakewatch catalog. There's a separate table per catalog, and each one contains only the signals produced by rules in that catalog. The table holds metadata about signals and the alerts they're associated with. Query it to analyze signals, correlate them with alerts, and build custom reports or dashboards.
Note
Signals are available in system tables, but alerts are not. To view and manage alerts, use the Alerts page in the UI.