Using floating eventhub consumer groups

Question

I have 10 instance of same application reading data from Eventhub, however I want to assign them separate consumer group so that they do not interfere in each other's read and process simultaneously. Each application maintaining its own checkpointing in its local storage.

If I assign a consumer group randomly to the 10 instances(at the time of deployment or update), can they still read the right data based on their local checkpoint.

[Something like floating consumer group but fixed checkpoints? I nowhere did I see checkpoint as a property of consumer group?]

Answer 1

Hi @Deeksha Thakur ,

I checked with internal team on this and below is the response from PG team. Kindly check.

So long as they know which partition and can specify a location in the event stream (offset/sequence number/timestamp), the consumer group shouldn't matter. There are a couple of caveats to be aware of:
• They won't be able to use one of our event processor types to read events without the consumer group becoming significant
• They won't be able to consume checkpoint data that our event processors have emitted without knowing the specific consumer group that was used
• They won't be able to request exclusive access to a partition unless they intend to prevent any other readers in that consumer group from accessing the partition
Outside of those considerations, I don't think there's a blocker to the proposed approach.

Hope this helps.

---------
Please consider hitting Accept Answer. Accepted answers helps community as well.