Eventhub offset inconsistent

Question

I am reading events from Azure periodically. I am not using Azure's checkpointing, instaed I keep track of the offset from the most recent read so the next call will start reading from the previously stored offset.

I noticed I can successfully read a few times but then the offset number suddenly goes down.

Is there a number defined in the documentation where offsets roll over? Would using sequence number be a more reliable approach?

Answer 1

@Atul Shaurya - Thanks for the question and using MS Q&A platform.

It sounds like you are experiencing an issue with inconsistent offsets when reading events from Azure Event Hubs. While there is no specific number defined in the documentation where offsets roll over, it is possible that you are encountering a scenario where the offset is being reset due to a failover or other event.

Using sequence numbers can be a more reliable approach to tracking the position in the event stream, as they are guaranteed to be unique and monotonically increasing. However, it's important to note that sequence numbers are not the same as offsets, and you will need to modify your code to use sequence numbers instead of offsets.

If you are still experiencing issues with inconsistent offsets, you may want to consider using Azure's checkpointing feature, which allows readers to mark or commit their position within a partition event sequence. Checkpointing can help ensure that you are always reading from the correct position in the event stream, even in the event of a failover or other disruption.

I hope this helps! Let me know if you have any further questions.