Object storage lifecycle in Unity Catalog

When you delete a Unity Catalog securable object (via Catalog Explorer, SQL DROP, etc.), what happens depends on the object type and storage type. This page describes the data file lifecycle, storage billing, and recovery options after deletion.

Storage type determines what happens to the data files

For tables and volumes, what happens to the underlying data files depends on whether the asset is managed or external. For more about this distinction, see Managed versus external assets in Unity Catalog.

  • Managed tables and volumes: Unity Catalog controls the storage location and data file lifecycle. Data files live in the managed storage location defined at the metastore, catalog, or schema level. When you delete a managed table or volume, Unity Catalog deletes the underlying data files through a multi-phase lifecycle. Managed storage locations come in two flavors:

    • Azure Databricks default storage: Object storage that Azure Databricks provisions and manages in your Azure Databricks account.
    • Customer-provided managed storage: A cloud storage location in your cloud account, configured at the metastore, catalog, or schema level, that Azure Databricks writes and manages data to.

    Both flavors share the same data file lifecycle, but billing and post-deletion file retention differ. See Managed objects on Azure Databricks default storage versus customer-provided storage.

  • External tables and volumes: You control the storage location and lifecycle. When you delete an external table or volume, Unity Catalog removes the metadata from the metastore, but the data files remain in your cloud storage location.

  • Foreign and federated catalogs: Data lives in another data source (such as a federated database through Lakehouse Federation, or a Hive metastore through Hive metastore federation). Unity Catalog holds only the connection metadata. When you delete a foreign catalog, Unity Catalog removes the connection metadata. Data in the source system is unaffected.

For other securable objects (catalogs, schemas, views, functions, models), deletion removes metadata only — there are no associated data files for Unity Catalog to manage. Dropping a catalog or schema with CASCADE removes the contained tables and volumes, each according to its own managed-or-external behavior above.

Recover a deleted object

How you recover a deleted object depends on the object type.

Object type Recovery
Tables, materialized views, streaming tables Use UNDROP within 7 days of the delete. Materialized views and streaming tables must have been created from an ETL pipeline and that pipeline must still exist.
Catalogs, schemas, volumes, views, functions, models Cannot be recovered after delete.

Warning

Recovery is time-limited and best-effort. Delete an object only after you confirm you no longer need the data. Use the RESTRICT option (the default) on DROP CATALOG and DROP SCHEMA to prevent accidental recursive deletion of non-empty objects.

Lifecycle of managed data after a delete

Deleting a managed table or volume does not immediately delete the data files from cloud storage. Data files are retained during a recovery window and permanently deleted afterward.

Phase 1: Recovery window

For 7 days after the delete, Unity Catalog retains the soft-deleted data so that you can recover the object. During this window:

  • Use the UNDROP SQL command to recover tables, materialized views, and streaming tables.
  • Unity Catalog retains the dropped object's metadata, and storage billing continues.

Phase 2: Purge

When the 7-day recovery window ends, the object can no longer be recovered. Unity Catalog permanently deletes the data files within 48 hours. For storage billing details in each phase, see Storage billing after a delete.

Managed objects on Azure Databricks default storage versus customer-provided storage

Managed objects can use two types of managed storage. The data lifecycle and Unity Catalog purge behavior are the same, but billing and post-purge file retention differ.

Storage type File purge
Azure Databricks default storage (managed by Azure Databricks on your behalf) Unity Catalog permanently deletes the data files within 48 hours of the recovery window ending.
Customer-provided managed storage (a cloud storage location you configured on the metastore, catalog, or schema) Unity Catalog permanently deletes the data files within 48 hours of the recovery window ending. Your cloud storage policies (object versioning, soft-delete, lifecycle rules) may retain files past this point. Your cloud provider bills you for the storage according to those policies.

For more about Azure Databricks default storage, see Default storage in Databricks. To configure customer-provided managed storage, see Specify a managed storage location in Unity Catalog.

External tables and volumes

When you delete an external table or external volume, Unity Catalog removes the metadata from the metastore. The data files in your cloud storage are not deleted. Your cloud provider continues to bill you for the storage according to your bucket's policies.

To remove the files, delete them directly from cloud storage.

Foreign and federated catalogs

A foreign catalog contains metadata that references an external data source. When you delete a foreign catalog, Unity Catalog removes the connection metadata. The data in the source system is unaffected. Azure Databricks does not bill you for storage in the source system; the source system's billing applies.

Storage billing after a delete

The following table summarizes how Azure Databricks and your cloud provider bill for storage in each phase. Azure Databricks only bills for storage on Azure Databricks default storage, and only during the recovery window — storage billing stops once the 7-day recovery window passes. For customer-provided managed storage and external storage, your cloud provider bills you directly.

Storage type Recovery window Phase 2 purge After purge
Managed on Azure Databricks default storage Azure Databricks billed Not customer-billable (billing stops once the recovery window passes) Not applicable
Managed on customer-provided storage Cloud provider billed Cloud provider billed Cloud provider billed (governed by your bucket's policies)
External tables and volumes Cloud provider billed (files are not deleted) Cloud provider billed Cloud provider billed
Foreign and federated catalogs Not applicable Not applicable Not applicable

After deleting a managed object on customer-provided storage, you might still see storage charges from your cloud provider. To reduce these charges, check your bucket's object versioning, soft-delete, and lifecycle policies.

Delete an object from Catalog Explorer

You can delete Unity Catalog objects from Catalog Explorer in the workspace UI. The data lifecycle described in this article applies whether you delete an object from Catalog Explorer or run a SQL DROP statement.

What happens when you delete a workspace

By default, deleting a workspace does not automatically delete the workspace's default Unity Catalog catalog. If the catalog is retained, its managed tables and volumes remain, and storage billing continues until the catalog is dropped.

Azure Databricks offers a force-delete option that removes the workspace catalog and its data as part of workspace deletion. See Force delete the workspace catalog. If the workspace catalog is not force-deleted, it is retained after workspace deletion and must be dropped manually.

For details about workspace deletion, see Delete a workspace.