Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Important
Lakewatch is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage workspace-level previews.
Lakewatch is a Databricks feature that brings security information and event management (SIEM) capabilities directly into your lakehouse. It helps security teams detect threats, investigate incidents, and respond to security events.
What does Lakewatch do?
Lakewatch gives security teams a single place to:
- Ingest security data from sources like CrowdStrike, Okta, AWS CloudTrail, and more, using API connectors, cloud storage paths, or existing tables in your workspace.
- Schematize data into governed tables with a consistent set of standard fields, using parsers, so you can write detection rules and queries that work across different sources of data.
- Detect threats with SQL-based detection rules that run on a schedule.
- Investigate incidents through alerts that automatically group related signals and link to the events and entities involved.
- Query security data with SQL directly in the Lakewatch UI, with support for time-range filtering, no-code refinements, and detailed row inspection.
- Investigate with AI using Genie: Genie Code, an assistant embedded across the Lakewatch UI, and Genie Agents for deeper, multi-step investigations. Use them to write queries, build detection rules, and triage alerts in natural language.
How it works
Lakewatch uses datasources to bring security data into your workspace. A datasource combines an ingestion method (how raw data is read) with one or more parsers that schematize the raw data into governed Unity Catalog tables:
- Raw data: Unprocessed data read from external sources by the datasource's ingestion method.
- Parsed tables: Schematized tables written by the datasource's parsers. Each parser describes the output columns of a log source and adds the Lakewatch-managed
lw_id,lw_meta, andlw_observablescolumns.
Parsed data is ready for security analysis:
- Detection rules run SQL queries against your parsed tables on a schedule. When a rule's conditions match, it generates a signal: a record of what was detected, when it happened, and which entities were involved.
- Lakewatch automatically correlates related signals into alerts for investigation. Analysts can assign, classify, and track alerts through resolution, and export them to Jira, ServiceNow, or Tines.
- The query page lets you run SQL queries across your security data for ad hoc investigation, with support for time-range filtering, no-code refinements, and detailed row inspection.
Lakewatch also monitors its own detection rules and datasources and opens system errors when they fail or have configuration problems. System errors are tracked separately from security alerts.
The Overview page
When you open Lakewatch, you land on the Overview page, your home page. It has three areas:
- A Genie question box at the top, under a greeting. Type a question, or select a suggestion such as What needs attention now? or Investigate most critical alert to fill in the box. Submitting the question opens Genie with it.
- An Activity last 7 days section with two panels: Data ingested, with your datasource count, and Signals, with your active detection rule count. Each panel compares the last 7 days with the preceding 7 days.
- Alerts and Errors tabs. Alerts shows how many security alerts are open, how many of those are unassigned, and the latest active alerts by severity. Errors shows the same summary for system errors.
In a new workspace, the panels start empty. Ingestion metrics fill in as you create datasources, signal metrics fill in as you create detection rules, and the alert cards fill in as detections generate alerts.
Additional resources
- End-to-end walkthrough: Walk through the core Lakewatch workflow.
- Ingest data: Create and manage datasources.