Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 37 ยท Time and Windows

Tumbling and Sliding Windows

Aggregate unbounded streams in time slices: Flink tumbling and sliding windows, window assigners, offsets, reduce, aggregate and ProcessWindowFunction, with runnable simulations.

You cannot sum an endless stream, but you can sum the last minute. A window cuts an unbounded stream into finite buckets so that normal aggregations make sense: orders per minute, average temperature per hour, unique visitors per day. Windows are the main reason event time and watermarks exist: they decide which bucket an event belongs to and when a bucket is complete.

This lesson covers the two most common window shapes, the window API with its three ways to compute a result, and the details of how windows are aligned and fired. It follows Watermarks.

Keyed and non-keyed windows

Java
// keyed: one independent window per key, runs in parallel
stream.keyBy(Click::page).window(assigner).aggregate(...);

// non-keyed ("all" windows): ONE window over the whole stream, parallelism 1
stream.windowAll(assigner).aggregate(...);

Prefer keyed windows. windowAll forces all data through a single subtask, which does not scale.

Tumbling windows

A tumbling window has a fixed size, and windows never overlap. Every event belongs to exactly one window.

flowchart LR
    W1["10:00 to 10:01"] --> W2["10:01 to 10:02"] --> W3["10:02 to 10:03"]

Tumbling windows of one minute: back to back, no overlap, no gap.

Java
clicks
    .keyBy(Click::page)
    .window(TumblingEventTimeWindows.of(Duration.ofMinutes(1)))
    .sum("count");

For processing time use TumblingProcessingTimeWindows. A tumbling window [start, end) includes its start and excludes its end: it contains timestamps start to end - 1 ms.

Alignment. Windows are aligned to the epoch, not to the first event. A 1-hour window always starts on the full hour in UTC. For a daily window in another time zone, give an offset: TumblingEventTimeWindows.of(Duration.ofDays(1), Duration.ofHours(-8)) for UTC+8. Without it, "per day" means UTC days.

The window containing timestamp ts starts at ts - (ts - offset + size) % size.

Sliding windows

A sliding window has a size and a slide. Windows overlap when the slide is smaller than the size, so one event can belong to several windows (size / slide of them).

Java
clicks
    .keyBy(Click::page)
    .window(SlidingEventTimeWindows.of(Duration.ofMinutes(10), Duration.ofMinutes(1)))   // "last 10 minutes, updated every minute"
    .sum("count");

Typical use: moving averages and "trailing N minutes" dashboards. Cost: each event is copied into size / slide windows, so a 1-hour window sliding by 1 second holds each event in 3,600 windows. Pick the largest slide you can accept, or use a ProcessFunction with state for very fine slides.

Stand-in: assigning events to windows

Java Stand-in: tumbling and sliding window assignment

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Each event is in exactly one tumbling window (59 vs 60 shows the half-open boundary) and in size / slide = 3 sliding windows.

Three ways to compute a window result

When the window fires, Flink gives you its content. You choose how to aggregate it:

1. reduce / aggregate: incremental and cheap. Flink keeps only one accumulator per window and updates it as each event arrives. State is tiny, regardless of how many events fall into the window.

Java
.reduce((a, b) -> new Count(a.page(), a.n() + b.n()))
// or the more general AggregateFunction<IN, ACC, OUT>
.aggregate(new AverageTemp())

2. process with ProcessWindowFunction: full access, but buffers all events. Flink stores every element of the window in state until it fires, then hands you the whole Iterable plus window metadata (start and end, key, current watermark). Needed for medians, top-N, or custom output that includes the window bounds.

Java
.process(new ProcessWindowFunction<Reading, String, String, TimeWindow>() {
    @Override
    public void process(String sensor, Context ctx, Iterable<Reading> readings, Collector<String> out) {
        long n = 0;
        for (Reading r : readings) { n++; }
        out.collect(sensor + " [" + ctx.window().getStart() + "," + ctx.window().getEnd() + ") count=" + n);
    }
})

3. Combine both: aggregate(aggFunction, processWindowFunction). The incremental function computes the result; the process function then receives a single pre-aggregated value and adds the window bounds. Best of both worlds.

Java
.aggregate(new AverageTemp(), new AddWindowBounds())
Approach State per window Access to all elements Use for
reduce / aggregate One accumulator No Counts, sums, averages, min/max
process All elements Yes Median, top-N, sorting
aggregate + process One accumulator No, but window metadata Aggregate plus window start/end

Stand-in: incremental versus buffered windows

Java Stand-in: per-minute counts with an incremental accumulator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The average needs only a sum and a count, so the incremental approach holds two numbers per window. The median needs every value, so it needs a buffering ProcessWindowFunction.

When does a window fire?

With event time, a window [start, end) fires when the watermark reaches end - 1. That is why a lagging watermark delays results. Processing-time windows fire when the machine's clock passes end. After firing, the window's state is cleared once the allowed lateness has passed, covered in Late Data and Allowed Lateness.

Empty windows do not fire: a key with no events in a minute produces no output for that minute. If you need a zero, emit it yourself with a KeyedProcessFunction and timers.

Common mistakes

  • Using windowAll on a high-volume stream and creating a single-subtask bottleneck.
  • Forgetting the watermark strategy, so event-time windows never fire.
  • Using a ProcessWindowFunction for a simple sum, which buffers every event for no reason.
  • Assuming daily windows follow the local time zone without an offset.
  • Sliding windows with a tiny slide, multiplying state and work.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A tumbling window of 1 minute has an event at exactly 10:01:00.000. Which window does it belong to?

    Windows include their start and exclude their end, so the event belongs to the window starting at 10:01.
  2. 2. In how many sliding windows (size 10 min, slide 1 min) does one event appear?

    size / slide = 10 windows overlap any point in time.
  3. 3. Which approach keeps the least state for computing a per-window average?

    An incremental AggregateFunction keeps a single small accumulator per window.
  4. 4. When does an event-time window [start, end) fire?

    The watermark promises that all events up to that time have arrived, so the window is complete.

Summary

  • Windows cut unbounded streams into finite buckets; prefer keyed windows so they run in parallel.
  • Tumbling windows do not overlap; sliding windows overlap and copy each event into size / slide windows; windows are aligned to the epoch, with offsets for time zones.
  • Use reduce/aggregate for small state, ProcessWindowFunction when you need all elements, or combine both to get window metadata cheaply.
  • Event-time windows fire on watermarks; empty windows produce no output.

Next: Session Windows, Global Windows and Triggers.