ADF and Kafka Connectivity

Kumar, PPrabhat 1 Reputation point
2022-12-05T13:14:35.447+00:00

Hi Team,

I am trying to export data from ADF to Kafka messaging queue. But I checked that I am unable to establish a connection between Azure Databricks and Kafka instance. Can you help me how to set up this?

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,947 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. AnnuKumari-MSFT 31,151 Reputation points Microsoft Employee
    2022-12-06T08:41:39.497+00:00

    Hi @Kumar, PPrabhat ,

    Welcome to Microsoft Q&A platform and thanks for posting your question here.

    As I understand your issue, you are trying to find the workaround to connect Databricks to Kafka . Please let me know if my understanding is incorrect.

    You can use the Databricks Apache Kafka connectors for Structured Streaming: The Apache Kafka connectors for Structured Streaming

    Write a Databricks notebook that extracts the data you require from Kafka and call it from ADF using a Databricks activity: Run a Databricks notebook with the Databricks Notebook Activity in Azure Data Factory

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    0 comments