How can I use MSSparkUtils in Python from PyCharm?

Victor Seifert 151 Reputation points
2022-05-10T08:49:10.1+00:00

I have a script that works in Synapse by utilizing the mssparkutils to retrieve credentials via a LinkedService:

storage_token_source = mssparkutils.credentials.getConnectionStringOrCreds(linked_service_name_source)

However, if I add this code to my local repository, PyCharm throws an Error as mssparkutils is unknown - I couldn't find the package in PyPi, how can I add it to the repository as an import/dependency?

I package my code later on and deploy it as a whl file to the spark pools.

Best regards,
Victor

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,668 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 85,346 Reputation points Microsoft Employee
    2022-05-10T13:14:04.393+00:00

    Hello @Victor Seifert ,

    Thanks for the question and using MS Q&A platform.

    You can download the mssparkutils package from below MVN repo and import the jar into your local project. With that, you should be able to compile the project successfully: https://mvnrepository.com/artifact/com.microsoft.azure.synapse/synapseutils

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators