Hi@Sai Birada ,
Sorry for the delay in response. We have heard back from internal team about your query.
Kafka Rest Proxy producer throughout depends on the configuration of brokers and Kafka management nodes. The shoebox metrics is available in Azure portal, to help you understand if Kafka management nodes is the bottleneck for your current load and configuration.
Kafka REST proxy has a very good performance result for Producing messages.
Throughput: up to 250MB/sec
• Kafka Broker: 10ms ~ 12ms
• Kafka REST proxy: 13ms~ 15ms
• Client E2E: 22~25ms
This is the result of the stress test of Kafka REST proxy which took under below configs.
• REST proxy requests - 15 batched messages of 1KB per request of Kafka REST proxy
• Producer config - 16KB batch size. Gzip compression
• Topic config - 4 partitions with 3 replicas
For producing messages, the bottleneck was Kafka REST proxy servers.
As a result, replication factors and the number of partitions did not affect the overall performance a lot.
Increasing the number of Kafka REST proxy servers increased its performance.
FYI, for consuming messages, the bottleneck was Kafka Brokers.
Please understand that performances get affected by the user's business use cases(traffic load) it may be different from the above results.
We encourage users to run perf tests with your anticipated workloads.
Hope this info helps.
Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.