I want to install python package in databrikcs job clusters and how to include this utility is "ini" file

Rohit Boddu 461 Reputation points

Hi Team,

How to install any python package in databricks jobs cluster ..

Requirement 1= and there is many 30 job clusters in my environment .. i dont want to install package individually in each job clusters is there any way to install package in all of job clusters in one go..

Requirement 2 = how to place this in "ini" file so whenever new cluster is created it automatically installs this utility ..

Thanks & Regards,

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,711 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Saurabh Sharma 21,196 Reputation points Microsoft Employee

    @Rohit Boddu Thanks for using Microsoft Q&A !!

    1. If all of your clusters are in same data-bricks workspace then you can use global init script either using the databricks UI or using Rest API. Please refer to the documentation - Add a global init script using the UI and Global Init Scripts API for details.
      You can add Global Init Scripts from UI by going to your "Admin Console" and +Add to add your required scripts - (See screenshots below)


    Please note that Only admins can create global init scripts.

    1. In order to place "init" file for a specific cluster, you need to put init scripts in a DBFS directory accessible by your cluster and then update the path under Advanced Options of the cluster. Please find below gif for you reference. Please refer to the documentation.



    Please do not forget to "Accept the answer" wherever the information provided helps you to help others in the community.