Hello @BATTINI Antoine ,
Thanks for the question and using MS Q&A paltform.
There are two primary ways to install a library on a cluster:
- Install a workspace library that has been already been uploaded to the workspace.
- Install a library for use with a specific cluster only.
In addition, if your library requires custom configuration, you may not be able to install it using the methods listed above. Instead, you can install the library using an init script that runs at cluster creation time.
As per your requirement, you can use init scripts.
Azure Databricks supports two kinds of init scripts: cluster-scoped and global.
- Cluster-scoped: run on every cluster configured with the script. This is the recommended way to run an init script.
- Global: run on every cluster in the workspace. They can help you to enforce consistent cluster configurations across your workspace.
For more details, refer to Azure Databricks cluster libraries and also check out different methods to install packages in Azure Databricks.
Hope this will help. Please let us know if any further queries.
------------------------------
- Please don't forget to click on
or upvote
button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
- Want a reminder to come back and check responses? Here is how to subscribe to a notification
- If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators