Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 37 ยท Time and Windows

Late Data and Allowed Lateness

Handle late events in Flink event-time windows: what makes an event late, allowedLateness and window updates, side outputs for dropped events, and choosing between delay and lateness.

However generous your watermark delay, some events will arrive after their window has already produced a result: a phone that came back online an hour later, a retried upload, a slow partition. These are late events. You have to decide what to do with them, and "ignore them quietly" is rarely the right answer.

Flink gives you three tools that work together: the watermark delay, allowed lateness and side outputs. This lesson explains each and how to combine them. It follows Session Windows, Global Windows and Triggers.

What makes an event late?

For an event-time window [start, end), the window fires when the watermark passes end - 1. An event is late for that window if it belongs to the window but arrives after the watermark has already passed the window's end. By default, Flink drops such events: the window has fired, its state may already be gone, and the result has been emitted.

Late data is a consequence of the trade-off you made in Watermarks: the smaller the delay, the earlier the results and the more events are late.

Tool 1: a bigger watermark delay

Raise forBoundedOutOfOrderness. Every window waits longer, so nearly all events are on time. The cost is latency for everyone, even when data is perfect. Good when most late events are only a few seconds late.

Tool 2: allowed lateness

Allowed lateness keeps a window's state around for an extra period after it has fired. If a late event arrives within this period, the window fires again with the updated result.

Java
clicks
    .keyBy(Click::page)
    .window(TumblingEventTimeWindows.of(Duration.ofMinutes(1)))
    .allowedLateness(Duration.ofMinutes(10))
    .sum("count");

Sequence for the window [10:00, 10:01) with 10 minutes of allowed lateness:

  1. The watermark passes 10:01: the window fires with the first result.
  2. A late event stamped 10:00:30 arrives at watermark 10:04: it is still within lateness, so it is added to the window and Flink fires again with a corrected result. Each late event that lands in the window triggers a new (update) firing.
  3. The watermark passes 10:11: the window is cleaned up. A later event for it is truly late and dropped (unless you use a side output).

Consequences you must handle:

  • The same window emits several results. Downstream must treat them as updates: use an upsert sink keyed by window and key, not an append-only one.
  • Window state lives allowedLateness longer, so state size grows with it.
  • Allowed lateness applies only to event-time windows; the default is 0.

Tool 3: side outputs for the rest

Events later than the allowed lateness can be diverted instead of vanishing. Tag the window with sideOutputLateData:

Java
OutputTag<Click> lateTag = new OutputTag<Click>("late-clicks") {};

SingleOutputStreamOperator<PageCount> counts = clicks
    .keyBy(Click::page)
    .window(TumblingEventTimeWindows.of(Duration.ofMinutes(1)))
    .allowedLateness(Duration.ofMinutes(10))
    .sideOutputLateData(lateTag)
    .aggregate(new CountClicks());

DataStream<Click> tooLate = counts.getSideOutput(lateTag);
tooLate.sinkTo(lateClicksSink);          // store for later correction or reconciliation

What you do with the late stream is a business decision: log and count it (a metric tells you whether your delay is right), write to a table that a nightly batch job uses to correct the numbers, or apply a correction downstream.

flowchart TB
    E["Event with timestamp t"] --> Q{"Window for t<br/>still open?"}
    Q -- "watermark before window end" --> A["Added; fires at window end"]
    Q -- "after end, within allowed lateness" --> B["Added; window fires again (update)"]
    Q -- "after end + lateness" --> C["Side output (or dropped)"]

Three fates of an event: on time, late but accepted, too late.

Stand-in: windows with allowed lateness and a late stream

Java Stand-in: on-time, late-but-accepted and too-late events

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Read the output from top to bottom: the first window fires when event d moves the watermark forward; e arrives within the lateness and corrects the count; once f moves the watermark far ahead, g has nowhere to go but the late stream.

Late data in processing-time pipelines

Processing-time windows have no late data in this sense, because they assign by arrival; that is both their simplicity and their weakness. If you see "late" handling being necessary, you are using event time.

Designing a lateness policy

  1. Measure first. Add a metric for the lag between event time and arrival time, and for the count of late events on the side output.
  2. Cover the bulk with the watermark delay, say the 95th to 99th percentile of lateness, as long as latency requirements allow.
  3. Cover the tail with allowed lateness if your sink can handle updates.
  4. Capture the rest with a side output for reconciliation, so numbers can be corrected in batch.
  5. Decide what counts as "good enough": dashboards can tolerate small errors; billing cannot.

This lambda-style fallback (fast streaming results, corrected later by a batch job over the stored late data) is a pragmatic pattern in many production systems.

Late data and timers

Event-time timers follow the same rules: a timer registered for a time at or below the current watermark fires immediately. In a KeyedProcessFunction, you can check ctx.timestamp() <= ctx.timerService().currentWatermark() to detect a late record and route it with ctx.output(...); see Process Functions.

Java Stand-in: detecting late records in a process function

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Note that an event with a timestamp equal to the watermark counts as late: the watermark already promised that nothing up to and including that time would arrive.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What happens by default to an event that arrives after its window has fired and been cleaned up?

    Without allowed lateness or a side output, late events are silently dropped.
  2. 2. What does allowedLateness(Duration.ofMinutes(10)) do?

    The window may fire again when late events arrive within the lateness period.
  3. 3. What must your sink support when you use allowed lateness?

    Repeated firings are refinements of the same window result.
  4. 4. How do you capture events that are later than the allowed lateness?

    sideOutputLateData routes them to a side output stream you can store or inspect.

Summary

  • A late event belongs to a window that has already fired; by default Flink drops it.
  • A bigger watermark delay trades latency for completeness; allowed lateness keeps window state after firing and re-fires with updates; side outputs capture what is still too late.
  • Sinks must handle updates when allowed lateness is on, and state grows with the lateness period.
  • Measure lateness before choosing a policy, and reconcile the late stream later if numbers must be exact.

Next: continue with the rest of the Apache Flink track.