Best way to execute multiple Synapse notebook activities in a pipeline

Question

Hi,

I can read here: https://learn.microsoft.com/en-us/answers/questions/672260/azure-synapse-use-single-session-for-multiple-note.html that it is not possible to use a single session for multiple Synapse notebooks in a pipeline. I haven't been able to find any documentation on whether this is still the case, or what to (optimally) do in a case where I have a pipeline and need to execute multiple Synapse notebook activities.

In the linked post they mention it is possible to execute one notebook with %run calls in the notebook of all the notebooks you want to simultaneously execute. Is this really the best way forward, or is there another way? Also, are there any general tips for how to optimally execute multiple notebooks, both simultaneously and successively?

Answer 1

Hello @Johan á Rogvi-Hansen ,

Thanks for the question and using MS Q&A platform.

Currently it’s not been supported yet - create one session in Synapse Analytics that multiple notebooks can use.

You can use %run and mssparkutils.notebook.run which executes multiple Synapse notebooks.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is jhow you can be part of Q&A Volunteer Moderators