Share via

Adding whl packages failing on DEP Enabled workspace.

Brian Maina 20 Reputation points
2023-10-18T03:43:18.67+00:00

Hello,

I am having some installation errors when trying to download great expectations and google-analytics data whl packages and the dependencies in my DEP enabled workspace.

I have followed this guide on installing whl packages Install third party whl packages I made some slight alterations since my spark pool is using Apache runtime 3.3. I used the latest miniforge3 shell since it comes with python=3.10 as the base and loaded a virtual environment with the Synapse-python310.yml file

I attached a picture of the whl packages and dependencies I generated for both great expectations and google analytics data.

This is the error i get when loading all the 19 whl packages. User's image

ERROR: Wheel 'jsonpointer' located at /usr/lib/library-manager/bin/lmjob/w2prdimdsynapse/wheels/jsonpointer-2.4-py2.py3-none-any.whl is invalid.

What i don't seem to understand is, I have a non-DEP enabled workspace, and when i put both great expectations and google-analytics-data in the requirements.txt file. The same dependencies install completely fine, according to the sttdout logs.

How can I address this error?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,697 questions
Accepted answer
  1. AnnuKumari-MSFT 32,161 Reputation points Microsoft Employee
    2023-10-31T10:08:46.91+00:00

    Hi Brian Maina ,

    Thankyou for putting effort and sharing the approach for resolving the issue.

    Glad that you figured out the way to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer.

    Uploading the wheel package to blob storage using blob fuse and retrieving them using Azure Storage Explorer caused changes in the MD5 hashes of the packages. By using drag and drop functionality within xrdp helped resolve the issue.

    Kindly accept the answer by clicking on Accept answer button so that the solution reaches the community in case anyone is facing similar issue. Thankyou.

    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Brian Maina 20 Reputation points
    2023-10-27T16:41:27.4266667+00:00

    I found the issue. I was making the packages within my Ubuntu instance Azure VM and load them to a blob storage via blob fuse. When i downloaded the packages from the blob using Azure storage explorer, it would change the MD5 hashes of the packages.

    I found out that i can drag and drop files within xrdp and avoid using blobfuse to transfer files outside the VM

    0 comments