list_files table-valued function

Applies to: check marked yes Databricks SQL check marked yes Databricks Runtime 18 LTS and above

Important

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Azure Databricks previews.

Returns the list of files at a path, including a FILE reference for each file.

Syntax

list_files(path [, connection => connection ] [, recursive => recursive ])

Arguments

  • path: A STRING path to list. Can be a volume path, an external location path, or a path on another file storage system such as SFTP, SharePoint, or Google Drive.
  • connection: An optional Unity Catalog connection used to authenticate with systems such as SFTP, SharePoint, or Google Drive.
  • recursive: An optional BOOLEAN that controls whether the function descends into subdirectories. Accepted values are:
    • true: Lists files in the path and all of its subdirectories, and doesn't return directories. This is the default value.
    • false: Lists only the immediate entries of the path, and returns directories with a null file reference and a size of 0.

The connection and recursive arguments must be passed by name using => syntax, for example connection => my_conn.

Returns

A table with one row per file, with the following columns:

Column Type Description
path STRING The path to the file.
size BIGINT The size of the file, in bytes.
modification_time TIMESTAMP The last modification time of the file, in UTC.
file FILE A reference to the file.

Common error conditions

  • LIST_FILES_AUTHORIZATION_ERROR.ON_PATH
  • LIST_FILES_AUTHORIZATION_ERROR.ON_CONNECTION
  • LIST_FILES_ERROR.PATH_NOT_EXISTS
  • LIST_FILES_ERROR.CONNECTION_NOT_EXISTS

For more information, see Error conditions in Azure Databricks.

Examples

To list the files in a volume directory:

SELECT * FROM list_files('/Volumes/my_catalog/my_schema/my_volume/');

The function returns one row per file:

path                                                 size   modification_time         file
/Volumes/my_catalog/my_schema/my_volume/file1.json   1234   2026-04-11 00:00:00.000   <file>

To list files from an external system using a Unity Catalog connection:

SELECT path, size FROM list_files('https://drive.google.com/drive/folders/my-folder-id', connection => my_gdrive_connection);

To list only the immediate entries of a directory, set recursive to false:

SELECT * FROM list_files('/Volumes/my_catalog/my_schema/my_volume/', recursive => false);

Directories appear in the results with a null file reference and a size of 0:

path                                                 size   modification_time         file
/Volumes/my_catalog/my_schema/my_volume/file1.json   1234   2026-04-11 00:00:00.000   <file>
/Volumes/my_catalog/my_schema/my_volume/subdir/      0      2026-04-11 00:00:00.000   NULL

To compute the total size of all files in a directory, sum the size column. Because recursive defaults to true, this includes files in subdirectories:

SELECT sum(size) AS total_bytes FROM list_files('/Volumes/my_catalog/my_schema/my_volume/');

The query returns the combined size in bytes:

total_bytes
1234
  • Last updated on