Integrate Apache Kafka with Event Hub

Question

Integrate Apache Kafka with Event Hub

satish manukonda 30

Working on POC to integrate confluence Kafka (data exists in topic) with Azure eventhub -

query - configuration changes need to done on confluence Kafka end apart from bootstrap changes/ user credential changes

Sander van de Velde | MVP 36,346 Reputation points MVP Moderator

2025-04-29T09:26:47.5766667+00:00

Hello @satish manukonda ,

welcome to this moderated Azure community forum.

Better questions with more information lead to better answers from our community members.

What is your question?
J N S S Kasyap 2,365 Reputation points Microsoft External Staff Moderator

2025-04-29T16:32:22.44+00:00

@satish manukonda

Just checking in to see if the below answer provided by @hossein jalilian helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

As comment mention by @Sander van de Velde | MVP If you provide the more information we will provide the well informed answer to your question.

Thanks

Accepted answer

0 additional answers

Your answer

Sander van de Velde | MVP 36,346 Reputation points MVP Moderator

2025-04-29T09:26:47.5766667+00:00

Hello @satish manukonda ,

welcome to this moderated Azure community forum.

Better questions with more information lead to better answers from our community members.

What is your question?
J N S S Kasyap 2,365 Reputation points Microsoft External Staff Moderator

2025-04-29T16:32:22.44+00:00

@satish manukonda

Just checking in to see if the below answer provided by @hossein jalilian helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

As comment mention by @Sander van de Velde | MVP If you provide the more information we will provide the well informed answer to your question.

Thanks

Answer 1

hossein jalilian 10,580 Moderator

Thanks for posting your question in the Microsoft Q&A forum.

The following configurations are required beyond bootstrap server and credential changes:

1- Security Protocol Settings:

Enforces SSL-encrypted communication with SASL authentication
Specifies the SASL mechanism for authentication

2- SASL JAAS Configuration: Must include the Event Hubs connection string as the password

3- Producer/Consumer-Specific Security Settings

4- Kafka Connect Configuration:

Internal topic configuration
Replication factor
Converters

5- Event Hubs-Specific Parameters:

Namespace and Hub Name
Consumer Group

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful

J N S S Kasyap 2,365 Reputation points Microsoft External Staff Moderator

2025-04-30T17:39:28.0333333+00:00

@satish manukonda
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
satish manukonda 30 Reputation points

2025-04-30T18:07:53.4666667+00:00
thanks a lot hossein jalilian for the steps ..

I replicated as below

installed docker desktop on local machine(windows x64)

started kafka in docker desktop

parallelly - started eventhub services in Azure

When I tried to integrate kafka which is running on docker desktop with Azure eventhub using below command

"kafka-console-producer --broker-list eventhubdemo-namespace.servicebus.windows.net --topic eventhubdemo-eventhub"

Issue :

Please provide valid host:port like host1:9091,host2:9092 when run below command "kafka-console-producer.sh \ --broker-list my-namespace.servicebus.windows.net:9093 \ --topic <eventhub-name> \ --producer.config client.properties"

Post that - I passed port and ran the above command as below

"kafka-console-producer --broker-list eventhubdemo-namespace.servicebus.windows.net:9093 --topic eventhubdemo-eventhub"

Again - it is failed for SASL -

Now created client.properties with SASL details

security.protocol=SASL_SSL

sasl.mechanism=PLAIN

sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \

username="$ConnectionString" \

password=" passed the details ";

ssl.endpoint.identification.algorithm=https

before I run below command

kafka-console-producer --broker-list eventhubdemo-namespace.servicebus.windows.net:9093 --topic eventhubdemo-eventhub --producer.config client.properties

I have below issues

Azure eventhub doesnt have explicit client.properties files

With out client.properties file - Kafka producer not able to connect to Azure eventhub

Now my Ask is - how i can integrate Kafka producer running in local machine (docker desktop) with Azure eventhub

Regards

Satish Manukonda
satish manukonda 30 Reputation points

2025-05-01T12:06:36.7133333+00:00

Hi All - thanks for wonderful support from this forum .. Able to complete my POC successfully .. we can close this thread

Share via

Integrate Apache Kafka with Event Hub

0 additional answers

Your answer