Here is our solutions
Single node Databricks spark cluster, install kafka on it and use debezium SQL connector to stream data, super cheap and easy solution, no need to waster money on HDinsight kafaka or Stream Sets or Event hubs complicated solutions, Kafka is open source and Debezium is open source.
SQL CDC Stream to Databricks
We have below scenario to be implemented for our Azure Setup; what options do i have in Azure
SQL DB -CDC Enabled-Stream CDC data continuously to Azure Data Bricks
Is it possible via Azure Event Hubs? Else we have to go with Kafka only? Else every minute job on spark engine itself?
Else go with any other third part tools?
Cost is big factory which plays into decision.