Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Important
Lakewatch is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage workspace-level previews.
Lakewatch provides an API connector to ingest data from CrowdStrike Falcon Event Streams. This article covers prerequisites, configuration instructions, connector schemas, and security observables.
Prerequisites
The connector requires:
- Serverless Lakeflow pipelines turned on in your workspace.
- Unity Catalog, with privileges to create or use a Unity Catalog connection. See Connection permissions.
- The Lakeflow Connect for CrowdStrike preview turned on in your workspace by a workspace admin. See Turn on the connector preview.
- If your workspace's serverless network policy uses restricted access, the source's API domains must be on the policy's Allowed domains list. See Allow the source API domains in your network policy.
- A Falcon administrator who can create an OAuth2 API client. See CrowdStrike's Event Streams API documentation.
- An API client with the Event streams scope and Read permission (
event_streamsread). The connector callsGET /sensors/entities/datafeed/v2and consumes the returned feeds.
Known limitations
- SCD type 2 is not supported, since Falcon Event Streams are append-only.
- Historical lookback isn't available for this connector. Ingestion starts at datasource creation. See Choose how far back to ingest.
Configure CrowdStrike for ingestion
CrowdStrike authenticates Falcon Event Streams requests with an OAuth 2.0 client credentials token. Databricks exchanges the client ID and client secret for an access token, then discovers and reads Falcon Event Streams feeds. For more information, see CrowdStrike's Event Streams API documentation.
- Sign in to the Falcon console as a Falcon administrator.
- Go to Support and resources, then API clients and keys.
- Click Add new API client.
- Enter a Client name (for example,
Event_Stream_Ingestor) and an optional Description. - Under API scopes, find Event streams and enable Read.
- Click Create.
- Copy the Client ID, Client Secret, and Base URL. CrowdStrike shows the client secret only one time. For the Databricks connection, enter the Base URL with the
https://scheme and no path (for example,https://api.crowdstrike.comorhttps://api.us-2.crowdstrike.com).
Tip
When creating a connection through the Lakewatch UI, you need the Client ID, Client Secret, and Base URL gathered above.
Next steps
After completing the configuration above, create a datasource with the CrowdStrike connector in the Lakewatch UI. See API connectors for instructions.
Connector and Lakewatch schemas
The connector ingests the following source table from the default source schema:
| Source table | CrowdStrike data | Built-in Lakewatch parser |
|---|---|---|
events |
Falcon Event Streams tenant events | crowdstrike_event_streams |
The connector writes each source table directly to the destination configured for its built-in parser. By default, the destination table is named <parser>_<datasource>, where <datasource> is the datasource name. For example, with the default datasource name crowdstrike_event_stream, the table is crowdstrike_event_streams_crowdstrike_event_stream. You can set a different destination during datasource creation. There is no separate raw staging table or later parsing step.
Lakewatch-managed columns
The connector derives:
time: A non-null timestamp frommetadata.eventCreationTime, interpreted as Unix epoch milliseconds. The connector usestimeas the cursor field for incremental ingestion.lw_id: A non-null string that Lakewatch uses as the primary key. The connector derives a stable, time-sortable ID fromtime, the event's feed, and its offset. The ID supports deduplication across incremental syncs.
For Lakewatch datasources, the connector also adds:
lw_meta: The datasource and connection identity, parser name and version, and ingestion time.lw_observables: Values identified by recursively scanning the event payload for security observables.
For more information, see Connector output tables.
Connector output table schema
The destination includes the source and event columns below, together with the lw_meta and lw_observables columns described above.
Schema
| Column | Data type |
|---|---|
lw_id |
STRING |
time |
TIMESTAMP |
event_data |
STRING |
data |
VARIANT |
Note
event_data is deprecated. Use data instead. data contains the same event information as event_data, but uses the VARIANT type for improved query performance.
Security observables
The connector scans data recursively for fields whose names identify the following observable types:
Observables
| Observable type | Matching field names |
|---|---|
| IP address | userip, localip, localaddress, remoteaddress |
| Hostname | computername, hostname, hostnamefield |
| Hash | md5string, sha1string, sha256string |
| MAC address | macaddress |
userid |
|
| Username | username |
| Trace ID | sessionid |
| URL | falconhostlink |
| MITRE ATT&CK technique | technique |