Error with Azure Synapse DEP-enabled workspaces.

Question

Error with Azure Synapse DEP-enabled workspaces.

Pranay 291

HI,

I have a synapse workspace with DEP-enabled. As pypi libraries cannot be installed directly in spark pool. I used the method, although the installation worked fine . I am still not able to use the REST API.

https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-manage-python-packages#install-wheel-files

Would be great if you could help.

Thanks

0 comments

Answer accepted by question author

0 additional answers

Your answer

Answer 1

PRADEEPCHEEKATLA 91,866

Hello @Pranay ,

Thanks for the question and using MS Q&A platform.

This is a Known limitation for synapse workspace with data exfiltration protection enabled.

Users can provide an environment configuration file to install Python packages from public repositories like PyPI. In data exfiltration protected workspaces, connections to outbound repositories are blocked. As a result, Python library installed from public repositories like PyPI are not supported.

As an alternative, users can upload workspace packages or create a private channel within their primary Azure Data Lake Storage account. For more information, visit Package management in Azure Synapse Analytics.

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Pranay 291 Reputation points

2021-11-10T08:07:39.027+00:00

@PRADEEPCHEEKATLA Thanks for your reply. As far as I can see the document this problem is only for Python pypi libraries.
I already tried using package management and it didn't work.
Another question is: will this also be a problem with Scala?? I haven't tried that yet, but just curious?

Thanks
PRADEEPCHEEKATLA 91,866 Reputation points

2021-11-11T07:06:30.793+00:00

Hello @Pranay ,

When you say "I already tried using package management and it didn't work", could you please share more details on the package which you have tired long with the stack trace of the error message?
PRADEEPCHEEKATLA 91,866 Reputation points

2021-11-15T08:26:58.787+00:00

Hello @Pranay ,

Follow up to see if you have had a chance to see the previous response. We need the following information to understand/investigate this issue further.
Pranay 291 Reputation points

2021-11-16T18:03:08.76+00:00

@PRADEEPCHEEKATLA : Hi Pradeep, thanks for your reply. I again tested it in a new synapse and its working fine. Although i had to later use REST API, which obviously didn't work.
I decided to go with a normal VNET connection.

Thank you
Pranay.
PRADEEPCHEEKATLA 91,866 Reputation points

2021-11-17T03:08:59.34+00:00

Hello @Pranay ,

Glad to know that it helped. Going forward continue to use MS Q&A platform for any questions related to Azure.

Share via

Error with Azure Synapse DEP-enabled workspaces.

0 additional answers

Your answer