Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
There are a variety of sample datasets provided by Azure Databricks and made available by third parties that you can use in your Azure Databricks workspace.
Unity Catalog datasets
Unity Catalog provides access to a number of sample datasets in the samples catalog. You can review these datasets in the Catalog Explorer UI and reference them directly in a notebook or in the SQL editor by using the <catalog-name>.<schema-name>.<table-name> pattern.
The nyctaxi schema (also known as a database) contains the table trips, which has details about taxi rides in New York City. The following statement returns the first 10 records in this table:
SELECT * FROM samples.nyctaxi.trips LIMIT 10
The tpch schema contains data from the TPC-H Benchmark. To list the tables in this schema, run:
SHOW TABLES IN samples.tpch
Third-party sample datasets in CSV format
Azure Databricks has built-in tools to quickly upload third-party sample datasets as comma-separated values (CSV) files into Azure Databricks workspaces. Some popular third-party sample datasets available in CSV format:
| Sample dataset | To download the sample dataset as a CSV file… |
|---|---|
| The Squirrel Census | On the Data webpage, click Park Data, Squirrel Data, or Stories. |
| OWID Dataset Collection | In the GitHub repository, click the datasets folder. Click the subfolder that contains the target dataset, and then click the dataset's CSV file. |
| Data.gov CSV datasets | On the search results webpage, click the target search result, and next to the CSV icon, click Download. |
| Diamonds (Requires a Kaggle account) | On the dataset's webpage, on the Data tab, on the Data tab, next to diamonds.csv, click the Download icon. |
| NYC Taxi Trip Duration (Requires a Kaggle account) | On the dataset's webpage, on the Data tab, next to sample_submission.zip, click the Download icon. To find the dataset's CSV files, extracts the contents of the downloaded ZIP file. |
To use third-party sample datasets in your Azure Databricks workspace, do the following:
- Follow the third-party's instructions to download the dataset as a CSV file to your local machine.
- Upload the CSV file from your local machine into your Azure Databricks workspace.
- To work with the imported data, use Databricks SQL to query the data. Or you can use a notebook to load the data as a DataFrame.
Third-party sample datasets within libraries
Some third parties include sample datasets within libraries, such as Python Package Index (PyPI) packages or Comprehensive R Archive Network (CRAN) packages. For more information, see the library provider's documentation.
- To install a library on an Azure Databricks cluster by using the cluster user interface, see Compute-scoped libraries.
- To install a Python library by using an Azure Databricks notebook, see Notebook-scoped Python libraries.
- To install an R library by using an Azure Databricks notebook, see Notebook-scoped R libraries.
Databricks datasets (databricks-datasets) mounted to DBFS
Azure Databricks recommends against using DBFS and mounted cloud object storage for most use cases in Unity Catalog-enabled Databricks workspaces. Some sample datasets mounted to DBFS are available in Azure Databricks
Note
The availability and location of Databricks datasets are subject to change without notice.
Browse DBFS mounted Databricks datasets
To browse these files from a Python, Scala, or R notebook, you can use Databricks Utilities (dbutils) reference. The following code lists all of the available Databricks datasets.
Python
display(dbutils.fs.ls('/databricks-datasets'))
Scala
display(dbutils.fs.ls("/databricks-datasets"))
R
%fs ls "/databricks-datasets"