Install Sentence-Transformers as a Workspace package on Azure Synapse Analytics

Question

I'm trying to install sentence-transformers python package as a workspace package for Azure Synapse Analytics.(To be used on Apache Spark Pool) Tried installing it through magic command %pip install sentence-transformers through a notebook and it works. Following the steps for uploading the sentence-transformers-2.2.2.tar.gz to Workspace and configuration throws the following error mentioned in the screenshot. I tested with couple of other python packages and they work fine. Any workarounds?

Answer 1

Haritha Thilakarathne - Thanks for the question and using MS Q&A platform.

PG Update(26/04/2023 2PM IST):
Currently, we are working on fix on improve this error message and the documentation with the changes.

The file .tar.gz format is only for R language packages (and not for python). To install python packages please use .yml or .whl file formats. 

26/04/2023 11 AM IST:
Looks like there is an issue with install whl file, which would leads to Job failed during run time with state=[dead] File not found. Please validate that file at the provided path exists and that you have the permissions required to access the file.
As per the repro, I had tried to install the sentence-transformers-2.2.2.tar.gz whl file via workspace packages where I got the same error message as shown above. I will reach out to the internal product team to get more details on why exactly we are experiencing this error message.

Wheanwhile, you can install the sentence-transformers library by providing a requirements.txt file.
Code snippet in requirement file: sentence-transformers==2.2.2

We had successfully installed the sentence-transformers library using requirements.txt file.

Notebook shows you how to use an already trained Sentence Transformer model to embed sentences for another task.

Hope this helps. Do let us know if you any further queries.