question

RyanAbbey-0701 avatar image
0 Votes"
RyanAbbey-0701 asked RyanAbbey-0701 commented

Synapse Notebook - dynamic name

Surely there must be an answer to this! I want to be able to have a configuration process that decides which notebook should be called in any particular instance - a similar process for configuring ingestion of files - but the notebook entry is not dynamic in that you have to explicitly state what notebook to run, anyone figured out how to set a notebook dynamically?

azure-synapse-analytics
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @RyanAbbey-0701 ,
Thanks for the ask and using the Microsoft Q&A platform .

At this time, we are reaching out to the internal team to get some help on this . We will update you once we hear back from them.
Thanks

Himanshu

0 Votes 0 ·
RuixinXu-0380 avatar image
1 Vote"
RuixinXu-0380 answered RyanAbbey-0701 commented

This is on the radar of Synapse Notebook team. We are going to start work on it since July. No ETA when it will be available on all regions yet.

· 7
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

That's slightly good news... but possibly too late for us to implement at client so probably have to look at DataBricks instead (since that does allow dynamic notebook selection)

0 Votes 0 ·

Sorry to hear that. I will circle back once this is live online. May I know which regions are you using?

0 Votes 0 ·

That's ok :)

Australia-East

1 Vote 1 ·

Any further news on this happening @RuixinXu-0380?

1 Vote 1 ·
Show more comments
SamaraSoucy-MSFT avatar image
0 Votes"
SamaraSoucy-MSFT answered RyanAbbey-0701 commented

There are two ways that you could implement this, though neither is setting a dynamic notebook name.

One is to use if/then or switch conditionals in the pipeline to determine which notebook to run:

110121-2021-06-29-01-21-08-analyticssynapsews-azure-synap.png

The second would be to create a notebook that handles the dynamic logic and calls the specific notebook needed in the pipeline run. Specifically, using `%run {notebook name} copies the contents of the other notebook copies everything from the referenced notebook into the one called by your pipeline and runs it as if it was part of the one in your pipeline. Any input parameters and output data are available to you.



· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Yes, both of those were thought about but not exactly user friendly

In particular
- for first suggestion: as I understand it, a pipeline has a limit to the number of activities and as we're expecting a large number of notebooks, we'd have to have a pipeline tree to get what we want
- for second suggestion: importing a notebook is not configurable and we'd therefore have to include every notebook within the "dynamic" notebook - good chance of introducing bugs

0 Votes 0 ·