Add a package in Synapse Spark pool but failed

Meng Zhang 31 Reputation points Microsoft Employee
2023-04-25T05:44:14.56+00:00

Hi, When I tried to import numba package, it failed with the error:
User's image

I found it is fixed in a new version. https://stackoverflow.com/questions/75094244/shap-systemerror-initialization-of-internal-failed-without-raising-an-except Therefore, I want to update numba. However, it failed no matter I used .txt or .whl.

.whl error: "ERROR: numba-0.56.4-cp310-cp310-win32.whl is not a supported wheel on this platform." .txt error: "ERROR: Cannot uninstall 'llvmlite'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall." And .txt file only has: numba==0.56.4 Can you help me import numba in my spark pool?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,572 questions
{count} votes

Accepted answer
  1. PRADEEPCHEEKATLA-MSFT 83,306 Reputation points Microsoft Employee
    2023-04-26T08:10:01.7933333+00:00


    Meng Zhang
    - Thanks for the question and using MS Q&A platform.

    PG Update(26/04/2023 2PM IST): Currently, we are working on fix on improve this error message and the documentation with the changes. The file .tar.gz format is only for R language packages (and not for python). To install python packages please use .yml or .whl file formats. 

    26/04/2023 11 AM IST:
    Looks like there is an issue with install whl file, which would leads to Job failed during run time with state=[dead] File not found. Please validate that file at the provided path exists and that you have the permissions required to access the file.
    As per the repro, I had tried to install the numba-0.56.4.tar.gz whl file via workspace packages where I got the same error message as shown above. I will reach out to the internal product team to get more details on why exactly we are experiencing this error message. User's image

    Wheanwhile, you can install the numba library by providing a requirements.txt file.
    Code snippet in requirement file: numba==0.56.4 User's image

    We had successfully installed the numba library using requirements.txt file. User's image

    Notebook shows Numba works well on code that looks like this: User's image

    Hope this helps. Do let us know if you any further queries.


0 additional answers

Sort by: Most helpful