Clarification on how to calculate watermark for Azure Streaming Analytics

Question

I'm working on understanding watermarks in Azure Streaming Analytics. Per MS documentation, there are two ways that watermarks can be calculated.

https://learn.microsoft.com/en-us/azure/stream-analytics/stream-analytics-time-handling#how-time-progresses-in-azure-stream-analytics

The second method states: "When there's no incoming event, the watermark is the current estimated arrival time minus the late arrival tolerance window. The estimated arrival time is the time that has elapsed from the last time an input event was seen plus that input event's arrival time."

Questions:

• What is meant by "No incoming events"? Does this mean that the source (ex. Event Hub) is verified to be empty?
• What is meant by "The last time an input event was seen."? Does this mean when it has exited the processing engine to the source?

Currently, this is how I interpret the calculation:

(7) Watermark = (5) [Estimated Arrival Time] - (6) [Late Arrival Tolerance Window]

----------

(5) Estimated Arrival Time =
(1) Elapsed Time = Time that elapsed from the last time an input event was last seen (2) and the current time (3) 
Plus
(4) Last Arrival Time

Answer 1

Hello @rwlindell,

Thanks for the question .

Yes "No incoming events" means that ideally there is nothing to process on the Eventhub .

For the second part , i think you have already gone through the doc which states

"When there's no incoming event, the watermark is the current estimated arrival time minus the late arrival tolerance window. The estimated arrival time is the time that has elapsed from the last time an input event was seen plus that input event's arrival time." .

If we are in the 0:45 mark and if you see that the expected events was at :40 unit ( lets assume that the
event is coming in every 5 unit of time ) . so the watermark will be (45-15 (6) in your example)

Thanks Himanshu

Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members