azure synapse analytics

Question

Hi i have uploaded the requirments.txt file and on running -

import pip #needed to use the pip functions
for i in pip.get_installed_distributions():
print(i)
there is no update on packages installed , I have recreated rescaled the machine but nothing seems to work

Answer 1

Hello @Sharma.ext, Nikhil ,

Welcome to Microsoft Q&A platform.

Could you please share the requirements.txt file to test from out end?

How to share the requirements.txt file in Microsoft & A platform?

You need to click on attachment symbol and add the text file from your computer.

Meanwhile, I would request you to go through the Official Document provide the guidance to add / manage libraries during the deployment in Apache Spark pool.

But in this article, we are going to discuss the steps, how the versions can be upgraded after the deployment of the Apache Spark Pool.

To list the current library version, you can use below code:

  import pip #needed to use the pip functions  
  for i in pip.get_installed_distributions(local_only=True):  
  print(i)  

And, you may checkout this thread which addressing similar issue.

Hope this helps. Do let us know if you any further queries.

----------------------------------------------------------------------------------------

Do click on "Accept Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

Answer 2

Hello @Sharma.ext, Nikhil ,

Not sure if this is just with the file whch you shared , but the file you shared has a filename which is missing a "e" , it should be "requirements.txt" . May be thats the issue .
I did tested the attached file and it worked for me ( please rename it to "requirements.txt") .

One more thing the spark pool user is required to have Storage Blob Data Contributor role on the linked ADLg2 storage account ( the one linked to the workspace , generally this is with the same name as the WS name ) .

[32396-requirements.txt]
1

Thanks Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members