Workspace package ModuleNotFoundError after successfully provisioning to Spark pool

Daniel King 5 Reputation points
2023-05-02T16:16:41.13+00:00

I have successfully uploaded a working python whl file as a package to Synapse and provisioned the package successfuly to my Spark Pool. However, when importing the package in a notebook running on my Spark Pool I'm receiving a ModuleNotFoundError. I have restarted the Spark Pool after provisioning and the custom package shows up in "!pip list" yet it cant seem to find the package when importing. Could this be an issue with the PATH the notebook kernel is looking in?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,554 questions
{count} vote

2 answers

Sort by: Most helpful
  1. Deleted

    This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.


    Comments have been turned off. Learn more

  2. PRADEEPCHEEKATLA-MSFT 82,271 Reputation points Microsoft Employee
    2023-05-04T04:32:16.92+00:00

    @Daniel King - Azure Synapse Runtime for Apache Spark 3.3 - google-auth comes default with Apache Spark version 3.3 - you need not to install this package seperately.

    User's image

    As per the repro, I had used Apache Spark pool 3.3 and verifed that the google-auth package has been already installed.

    User's image

    When you say getting ModuleNotFoundError error message- could you please help with the command which you are running along with the screenshot of the code and the error message?

    Hope this helps. Do let us know if you any further queries.


    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.