Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
The air command group within the Databricks CLI submits and manages GPU workloads, such as training and inference, with AI Runtime.
For workload configuration fields, see Workload YAML reference. To submit your first workload, see AI Runtime quickstart.
databricks air run
Submit a workload from a YAML configuration. The output includes the job run ID and job run URL.
databricks air run --file YAML_PATH [flags]
Arguments
None
Options
-f, --file string
Path to the workload YAML configuration. This option is required.
--watch
Stream logs from the first node (node 0) until the run reaches a terminal state. Default: false.
--override stringArray
Override a YAML field using FIELD_PATH=VALUE. Repeat this option to override multiple fields.
--dry-run
Validate the configuration without submitting a workload. Default: false.
--idempotency-key string
Return an existing run when the key was previously used. Overrides idempotency_token in the YAML. Maximum: 64 characters.
Examples
The following example submits a workload from train.yaml and streams logs from the first node (node 0) until the run finishes:
databricks air run --file train.yaml --watch
To submit in the background, omit --watch. The command returns after submission. Use the returned Job Run ID to check status and stream logs:
databricks air get <job-run-id>
databricks air logs <job-run-id>
Use the same workspace profile for submission, status, and logs. See Configuration profiles for the Databricks CLI.
Pass a configuration path as a separate argument to -h to display workload configuration help:
databricks air run -h config
databricks air run -h config.compute
databricks air run -h config.compute.accelerator_type
databricks air get
Get details for a run.
databricks air get JOB_RUN_ID [flags]
Arguments
JOB_RUN_ID
A job run ID returned by databricks air run.
Options
databricks air list
List runs. By default, the command lists active runs submitted by the current user.
databricks air list [flags]
Arguments
None
Options
--limit int
Maximum number of runs to show. Default: 20.
--all-status
Include runs in terminal states. Default: false.
--all-users
Include runs submitted by other users. Default: false.
--filter stringArray
Filter runs using KEY=VALUE. Repeat this option to add conditions. A run must match every condition. See Filter keys.
Filter keys
Each --filter option takes one KEY=VALUE condition. = is the only operator. You can't negate a condition, compare ranges, or combine conditions with OR.
| Key | Matches | Example |
|---|---|---|
accelerator_type |
The accelerator family, such as A10 or H100, or the full accelerator_type value from the workload YAML, such as GPU_8xH100. Family names are not case-sensitive. |
accelerator_type=H100 |
experiment |
A glob pattern compared to the entire experiment name. The match is not case-sensitive. * matches any sequence of characters, ? matches one character, and [abc] matches one character from the set. Regular expressions are not supported. A value with no wildcard matches only an experiment with exactly that name. |
'experiment=qwen*' |
num_accelerators |
The total number of GPUs for the run. The value must be a positive integer and must match exactly. | num_accelerators=8 |
user |
The username of the user who submitted the run. The value must match exactly. This key replaces the default restriction to your own runs, so you don't need --all-users. |
user=person@example.com |
For the supported accelerator_type values, see Accelerator types.
Filters narrow the runs that the command already lists. Without --all-status, only active runs can match.
Put single quotes around a condition that contains *, ?, or [ so that your shell passes the pattern to the CLI unchanged.
The command returns an error if a condition isn't in KEY=VALUE form, uses a key that isn't in the table, or has an empty or invalid value.
Examples
The following example lists your active runs that use H100 accelerators:
databricks air list --filter accelerator_type=H100
The following example lists runs in any state that person@example.com submitted, where the experiment name starts with qwen and the run uses eight GPUs in total:
databricks air list --all-status \
--filter 'experiment=qwen*' \
--filter num_accelerators=8 \
--filter user=person@example.com
databricks air logs
Stream or fetch logs for a run.
databricks air logs JOB_RUN_ID [flags]
Arguments
JOB_RUN_ID
A job run ID returned by databricks air run.
Options
--node int
Select a zero-based node index. Default: 0.
--tail int
Number of existing log lines to print. Default: 10000.
--minutes int
Fetch logs from the last number of minutes. Cannot be combined with --tail. Default: 0.
--retry int
Select a retry attempt. 0 is the initial attempt and -1 is the latest. Default: -1.
--download-to string
Download complete logs to a directory. Downloads all nodes unless --node is set. Cannot be combined with --tail or --minutes.
databricks air cancel
Cancel one or more runs by job run ID, or cancel all active runs submitted by the current user.
databricks air cancel [JOB_RUN_ID...] [flags]
Arguments
JOB_RUN_ID
One or more job run IDs. Cannot be combined with --all.
Options
--all
Cancel all active runs submitted by the current user. Default: false.
-y, --yes
Skip the confirmation prompt used by --all. Default: false.
databricks air images push
Push a container image to Artifact Registry.
For prerequisites and instructions for pushing an image, see Push images with the Databricks CLI.
databricks air images push [flags]
Arguments
None
Options
-s, --source string
The source container image as a name with an optional tag (NAME[:TAG]) or a digest (NAME@DIGEST). If omitted in an interactive terminal, the command prompts for the value.
--catalog string
The destination Unity Catalog catalog. If omitted in an interactive terminal, the command prompts for the value.
--schema string
The destination Unity Catalog schema. If omitted in an interactive terminal, the command prompts for the value.
--artifact string
The destination artifact as ARTIFACT[:TAG]. Defaults to the source image name and tag, or the latest tag when the source has no tag.
--pull
Pull the source image from its registry before pushing instead of reusing an available local image. Default: false.
databricks air convert-to-dabs
Convert a workload YAML configuration to a Databricks Asset Bundle. This local command writes databricks.yml and launch files under generated_artifacts/.
This command is an optional bridge to an advanced production workflow. You can continue to use databricks air run without converting the configuration.
For the bundle workflow, see Schedule GPU workloads and compose tasks.
databricks air convert-to-dabs YAML_PATH [flags]
Arguments
YAML_PATH
The path to the workload YAML configuration.
Options
--output-dir string
Directory where the command writes the bundle. The code source must be inside this directory. Default: the directory that contains the input YAML file.
--force
Overwrite generated bundle files that already exist. Default: false.
Global flags
--debug
Whether to enable debug logging.
-h or --help
Display help for the Databricks CLI or the related command group or the related command.
--log-file string
A string representing the file to write output logs to. If this flag is not specified then the default is to write output logs to stderr.
--log-format format
The log format type, text or json. The default value is text.
--log-level string
A string representing the log format level. If not specified then the log format level is disabled.
-o, --output type
The command output type, text or json. The default value is text.
-p, --profile string
The name of the profile in the ~/.databrickscfg file to use to run the command. If this flag is not specified then if it exists, the profile named DEFAULT is used.
--progress-format format
The format to display progress logs: default, append, inplace, or json
-t, --target string
If applicable, the bundle target to use
Migrate from the legacy Python CLI
The Databricks CLI and the deprecated Python-based air CLI use different command names and flags. Update scripts using the following mappings:
| Python CLI | Databricks CLI |
|---|---|
air run -f train.yaml |
databricks air run -f train.yaml |
air run -f train.yaml --override A=X B=Y |
databricks air run -f train.yaml --override A=X --override B=Y |
air get run <run-id> |
databricks air get <run-id> |
air list runs |
databricks air list --all-status |
air list runs --active |
databricks air list |
air -h config |
databricks air run -h config |
air logs <run-id> |
databricks air logs <run-id> |
air cancel <run-id> |
databricks air cancel <run-id> |