Azure Databricks documentation archive

Important

This documentation has been retired and might not be updated. The products, services, or technologies mentioned in this content are no longer supported.

In this archive, you can find earlier versions of documentation for Azure Databricks products, features, APIs, and workflows.

Administration

• Workspace-level SCIM provisioning

Compute

• Create cluster UI (legacy)
• Cluster UI preview
• Install a library with an init script (legacy)
• Cluster-named init scripts (legacy)
• Global init scripts (legacy)

Dev tools

• Manage libraries with %conda commands (legacy)
• Explore and create tables in DBFS
• Transactional writes to cloud storage with DBIO
• Browse files in DBFS
• FileStore
• Hive table (legacy)
• Koalas
• Skew join optimization using skew hints
• Legacy UniForm IcebergCompatV1
• Workspace libraries (legacy)
• Databricks CLI (legacy)
• What is dbx by Databricks Labs?
• dbutils.library
• Migrate to Spark 3.x
• VScode with Git folders
• VSCode workspace directory

Governance

• External metastores (legacy)
• Create Unity Catalog managed storage using a service principal (legacy)
• Credential passthrough (legacy)

Machine learning and AI

• Optimized LLM serving
• Model serving (legacy)
• Share feature tables across workspaces (legacy)
• MLeap ML model export
• Distributed training with TensorFlow 2
• Horovod
• Model inference using Hugging Face Transformers for NLP
• Train a PySpark model and save in MLeap format
• Set up and considerations for ai_generate_text()
• Analyze customer reviews with ai_generate_text() and OpenAI
• Apache Spark MLlib and automated MLflow tracking
• Load data using Petastorm

Release notes

• Databricks light
• Databricks runtime release notes (end-of-support)
• Unity Catalog GA release note

Repos and Git source control

• Git notebook versioning (legacy)

Security

• Audit log schemas for security monitoring
• Compute policies best practices

Storage

• Azure Blob storage file source with Azure Queue Storage (legacy)
• Azure Cosmos DB
• Structured Streaming writes to Azure Synapse
• Azure Event Hubs
• Connecting Azure Databricks and Azure Synapse with PolyBase (legacy)
• Neo4j
• Read and write XML data using the spark-xml library
• Accessing Azure Data Lake Storage Gen1 from Azure Databricks
• Configure Delta storage credentials
• Connect to Azure Blob Storage with WASB (legacy)