Synapse troubleshoot %run in live vs debug

Question

Hello,

I am having difficulty getting %run notebook to work in Synapse live for notebook not sharing the same path. I do not have problems when the path is the same.

I have tried no quotes and with quotes. I have tried path with a "/" in front and without a "/". The documentation suggests an absolute path begins with "/".

Each time the pipeline runs it's an error. Again it works fine when I debug a pipeline, but not in the live Synapse.

I'm really at a loss. Has anyone ever had success in Synapse live running a notebook in another notebook (to share variables?) with a different path?

To be clear, it works when I'm developing. It works in debug mode. IT DOES NOT WORK in live!!!

Here's the error:

Traceback (most recent call last)File ~/cluster-env/clonedenv/lib/python3.10/site-packages/IPython/core/magics/execution.py:696, in ExecutionMagics.run(self, parameter_s, runner, file_finder) 695 fpath = arg_lst[0] --> 696 filename = file_finder(fpath) 697 except IndexError as e: File ~/cluster-env/clonedenv/lib/python3.10/site-packages/IPython/utils/path.py:90, in get_py_filename(name) 89 return py_name ---> 90 raise IOError("File %r not found." % name) OSError: File 'sm/cc/cc_exceptions' not found. The above exception was the direct cause of the following exception: Exception Traceback (most recent call last)Cell In [5], line 1 ----> 1 get_ipython().run_line_magic('run', '"sm/cc/cc_exceptions"') File ~/cluster-env/clonedenv/lib/python3.10/site-packages/IPython/core/interactiveshell.py:2364, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth) 2362 kwargs['local_ns'] = self.get_local_scope(stack_depth) 2363 with self.builtin_trap: -> 2364 result = fn(args, **kwargs) 2365 return result File ~/cluster-env/clonedenv/lib/python3.10/site-packages/IPython/core/magics/execution.py:707, in ExecutionMagics.run(self, parameter_s, runner, file_finder) 705 if os.name == 'nt' and re.match(r"^'.'$",fpath): 706 warn('For Windows, use double quotes to wrap a filename: %run "mypath\myfile.py"') --> 707 raise Exception(msg) from e 708 except TypeError: 709 if fpath in sys.meta_path: Exception: File 'sm/cc/cc_exceptions' not found.

Answer 1

Hi @Jesse Horn-Artera
Thank you for the question and using Microsoft Q&A platform.

It seems like you are having trouble running a notebook in Synapse live that is not sharing the same path. Based on the error message you provided, it seems like the file 'sm/cc/cc_exceptions' is not found. To use %run magic command to reference another notebook within the current notebook's context, you need to provide an absolute path or notebook name only as a parameter. Relative path is not supported.
You mentioned that you have tried with and without quotes and with and without a "/" in front. Have you tried providing the full absolute path of the notebook you want to reference?

For example, if the notebook you want to reference is located at /path/to/notebook/Notebook1, you can use the following command to reference it:

%run /path/to/notebook/Notebook1

I have tried the above and was successful in running the pipeline without any errors:

I have 2 notebooks Notebook1 and Notebook2. I am calling Notebook1 from Notebook2 using the absolute path:

I have a pipeline which is executing Notebook2 using the notebook activity:

Here are a few steps and considerations that might help resolve the problem:

1. Publish Notebooks: Ensure that all notebooks involved are published. Unpublished notebooks might not be accessible in live mode.
2. Correct Path: Double-check the path you’re using. The path should be absolute and start with a / if it’s not in the same directory.
3. Use mssparkutils.notebook.run: Instead of %run, you can try using mssparkutils.notebook.run which might handle paths differently:

      from notebookutils import mssparkutils
      mssparkutils.notebook.run('/folder/notebook_name')
      
   

For more information please refer to the below links:
https://learn.microsoft.com/en-us/answers/questions/1233602/how-to-run-another-synapse-notebook-from-one-synap
https://learn.microsoft.com/en-us/answers/questions/512509/synapse-notebook-reference-how-to-use-run

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.