databricks vs ADF/synapse

arkiboys 9,186 Reputation points

I have started learning databricks...
It looks like databricks is more coding rather than ADF where it is mostly visual except where you add expressions, etc.
Am I right to assume that alot of the work carried out with databricks can be done with ADF or synapse workspace instead?

Thank you

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
8,486 questions
{count} votes

Accepted answer
  1. Nandan Hegde 27,316 Reputation points MVP

    Hey @arkiboys ,

    ADF is basically similar to SSIS wherein you have components inbuilt and you drag and drop and create flows.

    Whereas ADB(databricks) is completely code oriented wherein even to connect to source or sink you would need to write a code or to do a loop or iteration whereas in ADF it is a matter of creating a linked service directly and using For each loop.

    ADF is still an evolving tool so there are many scenarios which cannot be handled via ADF for which you need to write a custom code (there you can leverage Azure fucntions, Azure batch,Azure automation or databricks)

    So based on use cases, we take a call whether to leverage ADF or ADB

1 additional answer

Sort by: Most helpful
  1. Samy Abdul 3,361 Reputation points

    Hi @arkiboys , as Nandan has mentioned in prior post , Azure Data Factory(ADF) is a cloud orchestration tool for managing batch

    data integration workloads that runs your data pipelines (like ETL). The Azure Databricks provides batch processing solutions. ADF does not support

    streaming data natively and for that you have to rely on Event Hub etc. Below threads might be useful to go through. Thanks

    0 comments No comments