Script actions during cluster creation

Azure HDInsight on AKS provides a mechanism called Script Actions that invoke custom scripts to customize the cluster. These scripts are used to install additional components and change configuration settings. Script actions can be provisioned only during cluster creation as of now. Post cluster creation, Script Actions are part of the roadmap. This article explains how you can provision script actions when you create an HDInsight on AKS cluster.

Use a script action during cluster creation using Azure portal

1. Upload the script action in a ADLS/WASB storage(doesn't have to be the primary cluster storage). In this example we consider an ADLS storage. To upload a script into your storage, navigate into the target storage and the container where you want to upload it.

   

2. To upload a script into your storage, navigate into the target storage and the container. Click on the upload button and select the script from your local drive. After the script gets uploaded you should be able to see it in the container(see following image).

   

3. Create a new cluster as described here.

4. From the Configuration tab, select + Add script action.

   

   This action opens the Script Action window. Provide the following details:

   

   Property	Description
   Script Action Name	Unique name of the script action.
   Bash Script URL	Location where the script is stored. For example - abfs://<CONTAINER>@<DATALAKESTOREACCOUNTNAME>.dfs.core.windows.net/<file_path>, update the data lake storage name and file path.
   Services	Select the specific service components where the Script Action needs to run.
   Parameters	Specify the parameters, if necessary for the script.
   TimeOutInMinutes	Choose the timeout for each script

   

   Note

   • All the Script Actions persisted.
   • Script actions are available only for Apache Spark cluster type.

5. Select ‘OK’ to save the script.

6. Then you can again use + Add Script Action to add another script if necessary.

   

7. Complete the remaining cluster creation steps to create a cluster.

   Important

   • There's no automatic way to undo the changes made by a script action.
   • Script actions must finish within 40 minutes, or they time out causing cluster creation to fail.
   • During cluster provisioning, the script runs concurrently with other setup and configuration processes.
   • Competition for resources such as CPU time or network bandwidth might cause the script to take longer to finish.
   • To minimize the time it takes to run the script, avoid tasks like downloading and compiling applications from the source. Precompile applications and store the binary in Azure Data Lake Store Gen2.

View the list of Script Actions

1. You can view the list of Script Actions in the "Configuration" tab.