question

AlphonseOkossi-1577 avatar image
0 Votes"
AlphonseOkossi-1577 asked RaghuRamMahankaliQuadrantResource-8774 commented

Having issue installing mysql-connector-python package to Jupiter notebook on PySpark3 using Script Action-script failing

Here is my Script Action commands:

!/usr/bin/env bash

sudo /usr/bin/anaconda/bin/conda create --prefix /usr/bin/anaconda/envs/bgdatamysql python=3.5 anaconda --yes
sudo /usr/bin/anaconda/bin/conda install -c anaconda mysql-connector-python -n bgdatamysql --yes
sudo /usr/bin/anaconda/bin/conda install pandas -n bgdatamysql --yes
sudo /usr/bin/anaconda/bin/conda install panasql -n bgdatamysql --yes


Script action failing

azure-database-mysqlazure-hdinsight
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @AlphonseOkossi-1577,

Thanks for the ask and using the Microsoft Q&A platform.

In order to understand your issue - could you share the below details:

  • Could you please share the exact steps on where you are running the script action (Azure HDInsight cluster or Local machine) and how you are trying to execute script action?

  • When you say "Script action failing", are you seeing any error message?

  • In case if you find any error message, kindly share the complete stack trace of the error message which you are experiencing?

0 Votes 0 ·
  • I am running the script in HDInsight/Script Action:

  • List item

  • I was trying to create a new environment based on so the script I am trying to run is:

!/usr/bin/env bash

sudo /usr/bin/anaconda/bin/conda create --prefix /usr/bin/anaconda/envs/bgdatamysql python=3.5 anaconda --yes
sudo /usr/bin/anaconda/bin/conda install -c anaconda mysql-connector-python -n bgdatamysql --yes
sudo /usr/bin/anaconda/bin/conda install pandas -n bgdatamysql --yes
sudo /usr/bin/anaconda/bin/conda install panasql -n bgdatamysql --yes
- error:
Status
Failed

0 Votes 0 ·

Hello @AlphonseOkossi-1577,

Thanks for the details.

To known the exact reason for the error - you can SSH to the head node of the cluster and run sudo nano /var/lib/ambari-agent/data/output-682.txt.

0 Votes 0 ·

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered RaghuRamMahankaliQuadrantResource-8774 commented

Hello @AlphonseOkossi-1577,

Steps to install external Python packages using script action in Azure HDInsight:

Step1: Create a shell script.

  • Line2 - Creating Python virtual environment using condo

  • Line3 - Seaborn is the package name that you would like to install.

     #!/usr/bin/env bash
       sudo /usr/bin/anaconda/bin/conda create --prefix /usr/bin/anaconda/envs/py35new python=3.5 anaconda --yes
       sudo /usr/bin/anaconda/envs/py35new/bin/pip install seaborn
    

Step2: Upload your script Action

Upload your Script Action to a Storage Account that your chosen HDInsights have access to.

Bash script URI https://<my-storage-account>.blob.core.windows.net/<Storage-Container>/script-actions/<SctiptName>

Example:

 https://cheprahdistorage.blob.core.windows.net/chepra-2021-03-10t05-18-30-771z/script-actions/seaborn-package.sh

76129-image.png

Step3: Create a script action from Azure Portal.

  • Select the cluster that you want to run the Script action in Azure portal

  • On the blade under Configuration, you can find Script Actions

  • Select that and then click Submit New

  • Select custom

  • Give a name

  • Select nodes to install

76096-image.png

A few minutes Later, we have successfully installed external Python packages using script actions in Azure HDInsight.

76201-image.png

Hope this helps. Do let us know if you any further queries.


Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.


· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @AlphonseOkossi-1577,

Just checking in to see if the above answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

0 Votes 0 ·

Hello @AlphonseOkossi-1577,

Following up to see if the above suggestion was helpful. And, if you have any further query do let us know.
Take care & stay safe!

0 Votes 0 ·

Hi @PRADEEPCHEEKATLA-MSFT,

I followed the exact steps and here is the error message I see in head node.
I did sudo nano /var/lib/ambari-agent/data/output-186.txt to see the log.

Fetching package metadata ...........
Solving package specifications:
('Start downloading script locally: ', u'https://accountforhdinsigt.blob.core.windows.net/containerforscripts/create_ne$Fromdos line ending conversion successful
('Unexpected error:', "('Execution of custom script failed with exit code', 1)")
Removing temp location of the script

0 Votes 0 ·