The Synapse documentation states that the %%time magic command can be used in Spark notebooks in Synapse. This does not seem to be the case when calling those notebooks from a pipeline. Here's what I'm experiencing, using PySpark notebooks in Synapse pipelines:
If a notebook cell starts with %%time, it works in the notebook editor but the entire cell is bypassed when run from a pipeline. Bypassed, as in the output indicates that the prior cell runs, then the following cell, but the cell with %%time is just missing from the output!
If a notebook cell contains %time (just a single %), the notebook editor highlights it as incorrect syntax, but it still runs OK. When run from a pipeline, the cell generates an error.
MS, is this a bug, or is the Synapse documentation wrong and %%time is not supposed to work?