feature-engineering command group

Note

Databricks CLI use is subject to the Databricks License and Databricks Privacy Notice, including any Usage Data provisions.

The feature-engineering command group within the Databricks CLI allows you to manage features in your Databricks feature store.

databricks feature-engineering create-feature

Create a feature.

databricks feature-engineering create-feature FULL_NAME SOURCE INPUTS FUNCTION TIME_WINDOW [flags]

Arguments

FULL_NAME

    The full three-part name (catalog, schema, name) of the feature.

SOURCE

    The data source of the feature.

INPUTS

    The input columns from which the feature is computed.

FUNCTION

    The function by which the feature is computed.

TIME_WINDOW

    The time window in which the feature is computed.

Options

--description string

    The description of the feature.

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

Global flags

Examples

The following example creates a feature:

databricks feature-engineering create-feature my_catalog.my_schema.my_feature my_source my_inputs my_function my_time_window --description "My feature description"

databricks feature-engineering create-kafka-config

Create a Kafka config.

databricks feature-engineering create-kafka-config NAME BOOTSTRAP_SERVERS SUBSCRIPTION_MODE AUTH_CONFIG [flags]

Arguments

NAME

    Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name.

BOOTSTRAP_SERVERS

    A comma-separated list of host/port pairs pointing to Kafka cluster.

SUBSCRIPTION_MODE

    Options to configure which Kafka topics to pull data from.

AUTH_CONFIG

    Authentication configuration for connection to topics.

Options

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

Global flags

databricks feature-engineering create-materialized-feature

Create a materialized feature.

databricks feature-engineering create-materialized-feature FEATURE_NAME [flags]

Arguments

FEATURE_NAME

    The full name of the feature in Unity Catalog.

Options

--cron-schedule string

    The quartz cron expression that defines the schedule of the materialization pipeline.

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

--materialized-feature-id string

    Server-assigned unique identifier for the materialized feature.

--pipeline-schedule-state MaterializedFeaturePipelineScheduleState

    The schedule state of the materialization pipeline. Supported values: ACTIVE, PAUSED, SNAPSHOT.

Global flags

databricks feature-engineering create-stream

Create a Stream.

databricks feature-engineering create-stream NAME SOURCE_CONFIG CONNECTION_CONFIG SCHEMA_CONFIG INGESTION_CONFIG [flags]

Arguments

NAME

    Full three-part (catalog.schema.stream) name of the stream.

SOURCE_CONFIG

    Source-specific configuration. Determines the streaming platform source.

CONNECTION_CONFIG

    Specifies how to connect and authenticate to the stream platform.

SCHEMA_CONFIG

    Schema definitions for the stream. Currently only direct schemas are supported. In a future milestone, we will support schema registries through a UC Connection.

INGESTION_CONFIG

    Configuration for streaming data ingestion: the managed table storing an offline copy of forward fill data and optional historical backfill.

Options

--description string

    User-provided description.

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

Global flags

databricks feature-engineering delete-feature

Delete a feature.

databricks feature-engineering delete-feature FULL_NAME [flags]

Arguments

FULL_NAME

    Name of the feature to delete.

Global flags

Examples

The following example deletes a feature:

databricks feature-engineering delete-feature my_catalog.my_schema.my_feature

databricks feature-engineering delete-kafka-config

Delete a Kafka config.

databricks feature-engineering delete-kafka-config NAME [flags]

Arguments

NAME

    Name of the Kafka config to delete.

Global flags

databricks feature-engineering delete-materialized-feature

Delete a materialized feature.

databricks feature-engineering delete-materialized-feature MATERIALIZED_FEATURE_ID [flags]

Arguments

MATERIALIZED_FEATURE_ID

    The ID of the materialized feature to delete.

Global flags

databricks feature-engineering delete-stream

Delete a Stream.

databricks feature-engineering delete-stream NAME [flags]

Arguments

NAME

    Full three-part name (catalog.schema.stream) of the Stream to delete.

Global flags

databricks feature-engineering get-feature

Get a feature.

databricks feature-engineering get-feature FULL_NAME [flags]

Arguments

FULL_NAME

    Name of the feature to get.

Global flags

Examples

The following example gets a feature:

databricks feature-engineering get-feature my_catalog.my_schema.my_feature

databricks feature-engineering get-kafka-config

Get a Kafka config.

databricks feature-engineering get-kafka-config NAME [flags]

Arguments

NAME

    Name of the Kafka config to get.

Global flags

databricks feature-engineering get-materialized-feature

Get a materialized feature.

databricks feature-engineering get-materialized-feature MATERIALIZED_FEATURE_ID [flags]

Arguments

MATERIALIZED_FEATURE_ID

    The ID of the materialized feature.

Global flags

databricks feature-engineering get-stream

Get a Stream.

databricks feature-engineering get-stream NAME [flags]

Arguments

NAME

    Full three-part name (catalog.schema.stream) of the Stream to get.

Global flags

databricks feature-engineering list-features

List features.

databricks feature-engineering list-features [flags]

Options

--page-size int

    The maximum number of results to return.

--page-token string

    Pagination token to go to the next page based on a previous query.

Global flags

Examples

The following example lists all features:

databricks feature-engineering list-features

databricks feature-engineering list-kafka-configs

List Kafka configs.

databricks feature-engineering list-kafka-configs [flags]

Options

--limit int

    Maximum number of results to return.

--page-size int

    The maximum number of results to return.

Global flags

databricks feature-engineering list-materialized-features

List materialized features.

databricks feature-engineering list-materialized-features [flags]

Options

--feature-name string

    Filter by feature name.

--limit int

    Maximum number of results to return.

--page-size int

    The maximum number of results to return.

Global flags

databricks feature-engineering list-streams

List Streams.

databricks feature-engineering list-streams [flags]

Options

--limit int

    Maximum number of results to return.

--page-size int

    The maximum number of results to return.

--parent string

    Two-part name (catalog.schema) of the parent under which to list Streams.

Global flags

databricks feature-engineering update-feature

Update a feature's description (all other fields are immutable).

databricks feature-engineering update-feature FULL_NAME UPDATE_MASK SOURCE INPUTS FUNCTION TIME_WINDOW [flags]

Arguments

FULL_NAME

    The full three-part name (catalog, schema, name) of the feature.

UPDATE_MASK

    The list of fields to update.

SOURCE

    The data source of the feature.

INPUTS

    The input columns from which the feature is computed.

FUNCTION

    The function by which the feature is computed.

TIME_WINDOW

    The time window in which the feature is computed.

Options

--description string

    The description of the feature.

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

Global flags

Examples

The following example updates a feature's description:

databricks feature-engineering update-feature my_catalog.my_schema.my_feature description my_source my_inputs my_function my_time_window --description "Updated description"

databricks feature-engineering update-kafka-config

Update a Kafka config.

databricks feature-engineering update-kafka-config NAME UPDATE_MASK BOOTSTRAP_SERVERS SUBSCRIPTION_MODE AUTH_CONFIG [flags]

Arguments

NAME

    Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name.

UPDATE_MASK

    The list of fields to update.

BOOTSTRAP_SERVERS

    A comma-separated list of host/port pairs pointing to Kafka cluster.

SUBSCRIPTION_MODE

    Options to configure which Kafka topics to pull data from.

AUTH_CONFIG

    Authentication configuration for connection to topics.

Options

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

Global flags

databricks feature-engineering update-materialized-feature

Update a materialized feature.

databricks feature-engineering update-materialized-feature MATERIALIZED_FEATURE_ID UPDATE_MASK FEATURE_NAME [flags]

Arguments

MATERIALIZED_FEATURE_ID

    Server-assigned unique identifier for the materialized feature.

UPDATE_MASK

    Provide the materialization feature fields which should be updated. Currently, only the pipeline_state field can be updated.

FEATURE_NAME

    The full name of the feature in Unity Catalog.

Options

--cron-schedule string

    The quartz cron expression that defines the schedule of the materialization pipeline.

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

--materialized-feature-id string

    Server-assigned unique identifier for the materialized feature.

--pipeline-schedule-state MaterializedFeaturePipelineScheduleState

    The schedule state of the materialization pipeline. Supported values: ACTIVE, PAUSED, SNAPSHOT.

Global flags

databricks feature-engineering update-stream

Update a Stream.

databricks feature-engineering update-stream NAME UPDATE_MASK SOURCE_CONFIG CONNECTION_CONFIG SCHEMA_CONFIG INGESTION_CONFIG [flags]

Arguments

NAME

    Full three-part (catalog.schema.stream) name of the stream.

UPDATE_MASK

    The list of fields to update.

SOURCE_CONFIG

    Source-specific configuration. Determines the streaming platform source.

CONNECTION_CONFIG

    Specifies how to connect and authenticate to the stream platform.

SCHEMA_CONFIG

    Schema definitions for the stream. Currently only direct schemas are supported. In a future milestone, we will support schema registries through a UC Connection.

INGESTION_CONFIG

    Configuration for streaming data ingestion: the managed table storing an offline copy of forward fill data and optional historical backfill.

Options

--description string

    User-provided description.

--json JSON

    The inline JSON string or the @path to the JSON file with the request body.

Global flags

Global flags

--debug

  Whether to enable debug logging.

-h or --help

    Display help for the Databricks CLI or the related command group or the related command.

--log-file string

    A string representing the file to write output logs to. If this flag is not specified then the default is to write output logs to stderr.

--log-format format

    The log format type, text or json. The default value is text.

--log-level string

    A string representing the log format level. If not specified then the log format level is disabled.

-o, --output type

    The command output type, text or json. The default value is text.

-p, --profile string

    The name of the profile in the ~/.databrickscfg file to use to run the command. If this flag is not specified then if it exists, the profile named DEFAULT is used.

--progress-format format

    The format to display progress logs: default, append, inplace, or json

-t, --target string

    If applicable, the bundle target to use