mapping dataflow vs sparkpool

Question

Hi,
I understand spark can be used to do the same tasks as the mapping dataflows...
To me, (Using spark inside synapse workspace) is kind of similar to (Using c# inside ssis)
Questions:
1- Do you think if someone know mapping dataflows really well, then still need to learn spark?
2- Is my assumption correct in regards to spark in synapse being kind of similar to c# in ssis?

Any other thoughts?

Thank you

Answer 1

That's a pretty decent analogy. In Synapse pipelines (ADF), you can author a no-code data transformation job using Data Flows and you can write custom code in a Spark Notebook, then execute that from a control flow (pipeline) Notebook activity.