Utilize Kafka API through Azure Event Hubs VS Hosting Kafka in Azure Container Apps

Question

This is an "are we thinking about this correctly" type of question.

Currently: 3 AWS EC2 instances each running Kafka (0.10.2.0) and Zookeeper to support syncing between two on-premise CRMs via on-prem Windows services.

Problem: Administering the ec2 instances (monitoring, log cycling etc.) has taken a back seat, and therefore has caused stability issues (Lag sometimes spikes until we can figure out the issue(s)).

Desired Future State: A managed "Kafka"* solution that can serve as the message broker for a couple of services running on-prem that keep the two CRM instances in sync.

Question: Is there any advantage to us spinning up 3 instances of Kafka in Azure Container Apps, rather than just utilizing the Kafka API to interact with Azure Event Hubs?

References:

Validates we can use the Kafka API with Event Hubs: https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-for-kafka-ecosystem-overview

A lot of articles reference spinning up your own app in Azure Container Apps, that interact with Kafka, but none that are specifically hosting Kafka on Azure Container apps, which made me start questioning that route. e.g. https://quarkus.io/quarkus-workshops/super-heroes/index-azure.html

Also seeing Kafka referenced as infrastructure that can inform the scaling of apps themselves, which makes me think I should not be thinking of hosting Kafka as an app itself: https://azure.microsoft.com/en-us/services/container-apps/#features

**"Kafka" because our services are currently interacting with the Kafka API, but if we can use something else (like Eventhubs), which mirrors the Kafka API, then that's fine as well.*

tag: azure-container-apps (Tag not available?)

Answer 1

Reached out on the ACA Discord as well, and this was one response I received:

"You can't install Kafka in ACA since we don't support all the features it needs. Even if you get it to work, I still recommend using a managed service for this like EventHub with Kafka API or other managed Kakfa providers." - Tomer

https://discord.com/channels/938116342538706975/979098661466288148/979139137250680862

So Azure Event Hubs is the answer I am going with for now. If I receive a better answer here, I can reevaluate.