When does Azure Stream Analytics watermark calculation use 'no events arrived' formula

Question

When does Azure Stream Analytics watermark calculation use 'no events arrived' formula

Dhruv Singla 230

The azure documentation for time handling in azure stream analytics states the following.

When there's any incoming event, ...
When there's no incoming event, ...

I have the following doubts

In the statement 'When there's no incoming event', who determines the 'when'? Like is the watermark calculation done after every x seconds?
Suppose the last event which arrived at Azure Stream Analytics was 5 minutes ago. Would ASA use 'incoming event' formula or 'no incoming event' formula?

0 comments

Answer accepted by question author

1 additional answer

Your answer

Answer 1

Hi **Dhruv Singla,
**
Azure Stream Analytics decides when to use the “no incoming events” watermark formula based on the late arrival tolerance window, not on a fixed periodic timer and not on window definitions. While events are arriving, the watermark advances using event time (largest event time seen minus the out-of-order tolerance). When events stop arriving, the engine waits until the late arrival tolerance window elapses since the last observed event. After that point, ASA considers the input idle and advances the watermark using the current wall-clock time minus the late arrival tolerance window. In practice, if the last event arrived 5 minutes ago and the late arrival tolerance is smaller than that gap, ASA switches to the “no incoming event” formula and advances the watermark accordingly. This logic applies even if the query has no windows; windows only consume the watermark to decide when results can be finalized, they do not control when the watermark moves.

Reference Link :
Understand time handling in Azure Stream Analytics

Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

If you feel that your quires have been resolved, please accept the answer by clicking the "Upvote" and "Accept Answer" on the post.

SAI JAGADEESH KUDIPUDI 2,210 Reputation points Microsoft External Staff Moderator

2026-03-17T13:22:43.0366667+00:00

Hi Dhruv Singla,
Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 2

Marcin Policht 86,615 MVP Volunteer Moderator

The "when" in the phrase "When there's no incoming event" is determined by the internal event processing logic of Azure Stream Analytics (ASA). It doesn't depend on an arbitrary fixed time interval but instead on the event arrival pattern and the concept of a watermark.

The watermark in ASA is a mechanism that tracks the latest point in time that the system can process data reliably. The system uses this watermark to ensure that events arriving late can be handled appropriately. ASA maintains a watermark based on incoming events, typically advancing it when new events are processed. If there is a gap in incoming events (i.e., no event has arrived within the configured idle timeout), ASA will use the "no incoming event" logic to calculate the watermark. This typically happens after the configured idle timeout has passed, which is set using a TumblingWindow or HoppingWindow with the Watermark policy in ASA.

In your example, where the last event arrived 5 minutes ago, ASA will use the "no incoming event" formula if the idle timeout (the period where no new events have been received) exceeds the specified threshold. This timeout is configurable in the ASA job settings. So, if the configured idle timeout is, say, 2 minutes, and the last event was 5 minutes ago, ASA would treat this situation as "no incoming event" and apply the appropriate watermark calculation, marking the end of processing for the current time window.

The watermark calculation doesn't automatically happen after every fixed time period. Instead, it is driven by the absence of events and the specific configurations like event arrival rate, idle timeout, and windowing functions.

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin

Dhruv Singla 230 Reputation points

2026-03-08T14:31:32.68+00:00
Hi

Thanks for the response.

I got that there is something as 'idle timeout' that determines when to generate watermark. Can you give more details on the following points?

You said, "... configured idle timeout has passed, which is set using a TumblingWindow or HoppingWindow with the Watermark policy in ASA."

What is the 'Watermark' policy? Are you referring to the same thing about 'incoming events' and 'not incoming events'

What if the query doesn't have window? Then is it not defined?

You said, "This timeout is configurable in the ASA job settings."

Where is it configured? I didn't see any setting in the Azure UI.
Marcin Policht 86,615 Reputation points MVP Volunteer Moderator

2026-03-08T15:25:56.57+00:00

The watermark policy is designed to manage the timing and ordering of events in a streaming system, particularly when events arrive out of order or late. It allows the system to finalize processing of windows, such as those used in TumblingWindow or HoppingWindow, by determining a point in time up to which the system is confident that no more events will arrive with timestamps earlier than this value. The watermark is calculated as the minimum of the latest event time across all partitions, adjusted by the late arrival tolerance window. This helps ensure that windows can close and output their results, even when data is delayed.

The system handles two main scenarios: when events are actively coming in and when no events are arriving (idle). When events are coming in, the watermark moves forward based on the maximum event timestamp seen, minus the late arrival tolerance. If no events are coming in, the system generates a heuristic watermark based on the current wall-clock time, again minus the late arrival tolerance. This idle watermark calculation helps prevent the system from waiting indefinitely for late data, particularly in cases where partitions might stop sending events. When a partition becomes idle, its watermark will stall until the late arrival tolerance expires.

In queries that do not use any windowing functions like TumblingWindow or HoppingWindow, the watermark still plays a role in ensuring proper event ordering but does not directly control when time-based windows close. However, even without windowing, the watermark helps maintain the order of events, especially if you're using TIMESTAMP BY. This ensures that the system processes events based on their timestamps, even if the query is not explicitly using a time window.

From what I recall, to configure the late arrival tolerance window (which affects watermark behavior) in the Azure portal, you'd navigate to the Event Ordering settings in the Stream Analytics job configuration. You should be able to find there two key settings: the Out-of-order tolerance window and the Late arrival tolerance window. The late arrival tolerance defines the threshold for how long the system will wait for late events. The default value is usually 5 seconds, but this can be increased up to 20 days to prevent the watermark from stalling during periods of inactivity.

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin
Dhruv Singla 230 Reputation points

2026-03-09T05:33:09.2433333+00:00
Hi

Thanks for the response.

My main concern was regarding 'idle timeout' and not 'watermark'. Looks like there was a gap in my phrasing. Can you help me with the following?

As you said, 'idle timeout' is the time after which ASA updates the watermark. Now, how is this 'idle timeout' defined?

Is 'idle timeout' defined using 'late arrival' or 'out of order' settings? which one?

If 'idle timeout' is defined using 'window's, then do 'window' settings superseed the 'late arrival' or 'out of order' settings to determine 'idle timeout'? If 'window' are not defined, then how is 'idle timeout' determined?

Basically, who determines the 'when' in "when there are no incoming events"?
Marcin Policht 86,615 Reputation points MVP Volunteer Moderator

2026-03-09T14:38:32.35+00:00
The “when there are no incoming events” moment is determined by the Late Arrival Tolerance Window, not by window definitions and not by the Out-of-Order setting.

The engine advances the watermark when a new event arrives using the event timestamp. When no events arrive, the engine waits for the late arrival tolerance window to elapse since the last observed event time. Once that duration passes, ASA assumes no more events for that time will arrive and advances the watermark using the wall-clock time.

Conceptually the idle advancement behaves like:

watermark ≈ current_wall_clock_time - late_arrival_tolerance

The Out-of-Order tolerance controls how long ASA buffers events to reorder them when their timestamps arrive earlier than previously seen timestamps. It does not control idle behavior.

Window definitions such as:

TumblingWindow(...) HoppingWindow(...)

do not define or override the idle timeout. Windows only use the watermark to decide when a window can close. The watermark itself is still governed by the event arrival logic and the late arrival tolerance.

If a query has no window, the same watermark logic still runs internally, and the late arrival tolerance still determines when ASA considers a partition idle and advances time without new events.

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin
Dhruv Singla 230 Reputation points

2026-03-09T18:57:51.8133333+00:00

Hey

Thanks for the response. I finally got the clarity that I needed. Can you write a new answer or modify your exisiting one to include the following statement from your comment? I think it is the most clear and important of the answer.

"When no events arrive, the engine waits for the late arrival tolerance window to elapse since the last observed event time. Once that duration passes, ASA assumes no more events for that time will arrive and advances the watermark using the wall-clock time."
Marcin Policht 86,615 Reputation points MVP Volunteer Moderator

2026-03-09T19:12:17.0766667+00:00

As requested:

When no events arrive, the engine waits for the late arrival tolerance window to elapse since the last observed event time. Once that duration passes, ASA assumes no more events for that time will arrive and advances the watermark using the wall-clock time.

If the above response helps answer your question, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin
Marcin Policht 86,615 Reputation points MVP Volunteer Moderator

2026-03-11T12:18:40.6833333+00:00

Considering that you got the clarity you needed, remember to "Accept Answer" so that others in the community facing similar issues can easily find the solution. Your contribution is highly appreciated.

hth

Marcin

Share via

When does Azure Stream Analytics watermark calculation use 'no events arrived' formula

1 additional answer

Your answer