create one session in Synapse Analytics that multiple notebooks can use

Question

Hello,

It seems that a new Spark session is automatically created for every new notebook in Azure Synapse Analytics.

Is there a way to manually create one session and share it across multiple notebooks (in a pipeline for example)?

I found a similar question here with no answer

Thanks in advance!

Answer 1

Hello @Farrukh Tashpulatov ,

Thanks for the question and using MS Q&A platform.

Currently it’s not been supported yet.

Alternate method: I suggest you can try to use mssparkutils.notebook.run to see if it meets your requirement. Create a new notebook and put your ForEach logic into the new created notebook.

Python code cell as example:

   for notebook in notebookNames:  
          mssparkutils.notebook.run(notebook)  

For more details, refer to Introduction to Microsoft Spark utilities - Azure Synapse Analytics | Microsoft Learn

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators