question

FarrukhTashpulatov-8468 avatar image
0 Votes"
FarrukhTashpulatov-8468 asked FarrukhTashpulatov-8468 commented

create one session in Synapse Analytics that multiple notebooks can use

Hello,

It seems that a new Spark session is automatically created for every new notebook in Azure Synapse Analytics.

Is there a way to manually create one session and share it across multiple notebooks (in a pipeline for example)?

I found a similar question here with no answer

Thanks in advance!


azure-synapse-analytics
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

PRADEEPCHEEKATLA-MSFT avatar image
0 Votes"
PRADEEPCHEEKATLA-MSFT answered FarrukhTashpulatov-8468 commented

Hello @FarrukhTashpulatov-8468,

Thanks for the question and using MS Q&A platform.

Currently it’s not been supported yet.

Alternate method: I suggest you can try to use mssparkutils.notebook.run to see if it meets your requirement. Create a new notebook and put your ForEach logic into the new created notebook.

Python code cell as example:

    for notebook in notebookNames:
           mssparkutils.notebook.run(notebook)

For more details, refer to Introduction to Microsoft Spark utilities - Azure Synapse Analytics | Microsoft Docs

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

this is great,
thanks a lot!

1 Vote 1 ·