cacheTable

Caches the specified table in-memory or with given storage level. Default MEMORY_AND_DISK.

Syntax

cacheTable(tableName: str, storageLevel: StorageLevel = None)

Parameters

Parameter	Type	Description
tableName	str	Name of the table to get. Can be qualified with catalog name.
storageLevel	StorageLevel, optional	Storage level to set for persistence.

Notes

Cached data is shared across all Spark sessions on the cluster.

Examples

_ = spark.sql("DROP TABLE IF EXISTS tbl1")
_ = spark.sql("CREATE TABLE tbl1 (name STRING, age INT) USING parquet")
spark.catalog.cacheTable("tbl1")

# or

spark.catalog.cacheTable("tbl1", StorageLevel.OFF_HEAP)

# Throw an analysis exception when the table does not exist.
spark.catalog.cacheTable("not_existing_table")
# Traceback (most recent call last):
#     ...
# AnalysisException: ...

# Using the fully qualified name for the table.
spark.catalog.cacheTable("spark_catalog.default.tbl1")
spark.catalog.uncacheTable("tbl1")
_ = spark.sql("DROP TABLE tbl1")