How to Consume from eventhub at constant speed?

Question

I am consuming events from eventhub and sending those events to a microservice having REST API. I want to put constant tps on the API which I am hitting even though there are lot of events pending to be consumed in eventhub.
Even if I consume in BATCH I am not achieving this as its is either size or time based and not both at a time.

Eg. I want to send events to another API by consuming 5 events per second from each partition.

Using spring-cloud-stream-binder.
spring-cloud-azure-stream-binder-eventhubs

Answer 1

Hello @Chaitragandha Patil ,
Thanks for the question and using MS Q&A platform.

As I understand, you want logic to control the rate at which events are consumed from EventHub to avoid overwhelming downstream components.

The below is pseudocode describing the logic, not actual copy-paste code.

interval_time = 5 #desired time between events  
stop_signal = false #to escape loop  

do until stop_signal is true:  
    start_time = gettime()  
    next_time = start_time + interval_time  
    consume_events_synchronously()  
    now_time = gettime()  
    if now_time < next_time:  
        wait( next_time - now_time )  
endloop  

In the above, we make a time check before consuming more events. This makes our code wait if it runs too quick. However if the event consumption takes too long, there is no recovery of lost time. This means we run at a rate at least as slow as interval, but could run slower.

Please do let me if you have any queries.

Thanks
Martin

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators