Events
Mar 31, 11 PM - Apr 2, 11 PM
The ultimate Microsoft Fabric, Power BI, SQL, and AI community-led event. March 31 to April 2, 2025.
Register todayThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
An init script (initialization script) is a shell script that runs during startup of each cluster node before the Apache Spark driver or executor JVM starts. This article provides recommendations for init scripts and configuration information if you must use them.
Databricks recommends using built-in platform features instead of init scripts whenever possible. Widespread use of init scripts can slow migration to new Databricks Runtime versions and prevent adoption of some Databricks optimizations.
Important
If you need to migrate from init scripts on DBFS, see Migrate init scripts from DBFS.
The following Databricks features address some of the common use cases for init scripts:
If you must use init scripts:
The following table provides recommendations organized by Databricks Runtime version and Unity Catalog enablement.
Environment | Recommendation |
---|---|
Databricks Runtime 13.3 LTS and above with Unity Catalog | Store init scripts in Unity Catalog volumes. |
Databricks Runtime 11.3 LTS and above without Unity Catalog | Store init scripts as workspace files. (File size limit is 500 MB). |
Databricks Runtime 10.4 LTS and below | Store init scripts using cloud object storage. |
Azure Databricks supports two kinds of init scripts: cluster-scoped and global, but using cluster-scoped init scripts are recommended.
Whenever you change any type of init script, you must restart all clusters affected by the script.
Global init-scripts run before cluster-scoped init scripts.
Important
Legacy global and legacy cluster-named init scripts run before other init scripts. These init scripts are end-of-life, but might be present in workspaces created before February 21, 2023. See Cluster-named init scripts (legacy) and Global init scripts (legacy).
You can store and configure init scripts from workspace files, Unity Catalog volumes, and cloud object storage, but init scripts are not supported on all cluster configurations and not all files can be referenced from init scripts. For environment recommendations for init scripts, see Recommendations for init scripts.
The following table indicates the support for init scripts based on the source location and the cluster access mode. The Databricks Runtime version listed is the minimum version required to use the combination. For information about cluster access modes, see Access modes.
Note
Shared access mode requires an admin to add init scripts to an allowlist
. See Allowlist libraries and init scripts on shared compute.
Shared access mode | Single access mode | No-isolation shared access mode | |
---|---|---|---|
Workspace files | Not supported | All supported Databricks Runtime versions. On versions below 11.3 LTS, init scripts that reference other workspace files are not supported. |
All supported Databricks Runtime versions. |
Volumes | 13.3 LTS and above | 13.3 LTS and above | Not supported |
Cloud storage | 13.3 LTS and above | All supported Databricks Runtime versions | All supported Databricks Runtime versions |
Warning
Init scripts on DBFS have reached end-of-life and can no longer be used. You must migrate your init scripts to a supported location before you can start compute. Store init scripts on Unity Catalog Volumes, as workspace files, or in cloud object storage.
Users that need to migrate init scripts from DBFS can use the following guides. Make sure you’ve identified the correct target for your configuration. See Recommendations for init scripts.
Events
Mar 31, 11 PM - Apr 2, 11 PM
The ultimate Microsoft Fabric, Power BI, SQL, and AI community-led event. March 31 to April 2, 2025.
Register todayTraining
Module
Automate workloads with Azure Databricks Jobs - Training
Automate workloads with Azure Databricks Jobs
Certification
Microsoft Certified: Azure Database Administrator Associate - Certifications
Administer an SQL Server database infrastructure for cloud, on-premises and hybrid relational databases using the Microsoft PaaS relational database offerings.