Skip to content
Tutorials Center
My learning

Topics

Lesson 40 of 79 ยท Transport Layer

TCP Flow Control and the Receive Window

Understand TCP's sliding window and receive window, window scaling, zero-window probes, silly window syndrome, Nagle's algorithm and delayed ACKs, with simulations.

A fast sender can easily overwhelm a slow receiver. Imagine a server pushing a large file to a phone whose app only reads data a little at a time. The phone's operating system keeps incoming bytes in a receive buffer, and if that buffer fills up, new data has nowhere to go and must be dropped. Flow control prevents that: the receiver tells the sender how much free buffer space it has, and the sender never sends more than that.

This is not the same as congestion control, which protects the network between the two hosts. Flow control protects the receiver. In this lesson you learn the sliding window and the receive window, why the 16-bit window field needed an extension, what happens when the window reaches zero, and two sender and receiver habits (Nagle's algorithm and delayed ACKs) that exist to avoid tiny segments but can clash with each other.

The sliding window

You met sliding windows at the link layer in flow control and ARQ. TCP uses the same idea but counts bytes. The sender's view of the byte stream splits into four parts:

flowchart LR
    A["Sent and<br/>acknowledged"] --> B["Sent, not yet<br/>acknowledged<br/>(in flight)"] --> C["Allowed to send<br/>now"] --> D["Not allowed<br/>yet"]

The window is the in-flight part plus the part the sender may still send. As ACKs arrive the whole window slides to the right.

Every segment the receiver sends carries two things that matter here:

  • the acknowledgment number: the next byte it wants (everything before is safely received), and
  • the window field: the number of additional bytes it is willing to accept, starting at that acknowledgment number. This is the receive window, abbreviated rwnd.

The sender may have data outstanding up to the byte ack + window - 1. In other words, bytes in flight <= rwnd. When an ACK moves the left edge forward and the window stays the same, the right edge moves forward too, which lets more data out. That is the "sliding".

The window changes dynamically. If the application reads data quickly, free space stays large, and so does the advertised window. If the application is slow, the buffer fills, and the window shrinks. The receiver should not shrink the window in a way that withdraws space it has already promised (the right edge must not move backwards).

Python A receive window shrinking to zero and reopening

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the table row by row. The window the sender used in a round is what the receiver advertised at the end of the previous one; the buffer column shows what is left unread after the application has taken its share. Whenever the application pauses, the buffer fills and the window collapses. When it resumes, the window opens again.

Zero windows and window probes

If the buffer is full, the receiver advertises a window of 0. The sender must stop sending data. There is a danger here: the receiver will later send a window update to say "I have space again", but that update is a pure ACK, and pure ACKs are not retransmitted. If it is lost, both sides wait for each other forever.

TCP avoids that deadlock with the persist timer. While the window is zero, the sender periodically sends a tiny window probe (one byte of data or a bare ACK) and the receiver answers with its current window. The probes are spaced with exponential backoff, like retransmissions. A connection shown in a capture with "TCP ZeroWindow" and "TCP Window Update" is waiting on a slow application, not a slow network. That distinction matters when you troubleshoot slow transfers.

The window and the speed limit

The window also limits throughput. In one round trip time the sender can send at most one window of data before it must wait for an ACK, so:

Text
maximum throughput = window / RTT

With a 64 KB window and a 100 ms round trip, that is about 5.2 megabits per second, no matter how fast the links are. The amount of data needed to keep a path full is the bandwidth-delay product, which you met in bandwidth, latency and loss.

Window scaling

The window field is only 16 bits wide, so it can express at most 65,535 bytes. That was fine in 1981 and is a severe limit on fast, long paths. The window scale option, sent in the SYN and SYN-ACK, fixes it: each side announces a shift count from 0 to 14, and the real window is the field value shifted left by that many bits (multiplied by 2^shift). The largest window becomes 65,535 x 2^14, about 1 GiB. The option is negotiated only in the SYN, and it applies in both directions only if both sides include it. Each side's shift is independent and fixed for the connection.

Python Window limits throughput; window scaling lifts the limit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Modern operating systems also autotune their receive buffers, growing them as the connection needs, so a large window can be advertised only when the path really calls for it.

Silly window syndrome

Flow control has an infamous failure mode. Suppose the receiver's buffer is full and the application then reads only a few bytes at a time. If the receiver advertises every small bit of free space immediately, the sender will answer with tiny segments: 100 bytes of data with 40 bytes of headers is a waste. The connection ends up exchanging a stream of nearly empty packets even though plenty of bandwidth is available. This is the silly window syndrome (SWS).

Both sides have a rule to avoid it:

  • Receiver (Clark's algorithm): do not advertise a larger window until the free space is at least min(MSS, buffer / 2). Until then, keep advertising 0.
  • Sender: do not send a small segment just because a small window opened; wait until a full MSS, or at least half of the largest window the receiver has ever advertised, can be sent. (Nagle's algorithm, below, is the sender's rule for the small writes of the application.)
Python Silly window syndrome and Clark's fix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Nagle's algorithm

The sender has its own small-packet problem. An interactive program such as a remote terminal writes one byte at a time. Sending each byte in its own segment means 40 bytes of header for 1 byte of data. Nagle's algorithm limits this with one simple rule:

If there is unacknowledged data in flight, hold new small writes in the buffer. Send when the outstanding data is acknowledged, or when enough has accumulated to fill a full segment.

On a fast network ACKs return quickly, so little is delayed. On a slow link, many small writes are merged into a single segment, which is exactly what you want. A connection with nothing in flight sends immediately, so a single small request is never held back.

Python Nagle's algorithm merges small writes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Delayed ACK meets Nagle

The receiver has a matching habit, the delayed ACK from the reliability lesson: wait up to about 40 ms in the hope of piggybacking the ACK on response data or acknowledging two segments at once. Each habit is sensible alone. Together they can produce a nasty stall.

Consider a client that writes a request in two small writes (a header and then a body). The first goes out at once. The second is held by Nagle because the first is unacknowledged. The server, meanwhile, has received only half a request and has nothing to answer, so it delays its ACK. The client's second write waits for that ACK. After around 40 ms the delayed-ACK timer fires, the ACK is sent, the client sends the rest, and the exchange finally proceeds. Every request pays 40 ms for nothing.

The usual fixes:

  • Write once. Combine the pieces into one buffer (or use gathering writes) so the whole request is a single send.
  • Disable Nagle with the TCP_NODELAY socket option for latency-sensitive traffic such as interactive protocols, games or RPC with many small messages. Many HTTP servers and libraries set it by default.
  • Quick ACK options on the receiver exist in some systems, but the first two are cleaner.

Do not turn Nagle off to "make things faster" without a reason. If your program writes many tiny pieces separately, TCP_NODELAY makes it send many tiny packets. First reduce the number of writes; use TCP_NODELAY for what is truly interactive.

Flow control and congestion control together

The sender must obey two limits at the same time: the receiver's window rwnd and its own congestion window cwnd, which is an estimate of what the network can carry. The amount in flight is bounded by the smaller of them:

Text
bytes in flight <= min(rwnd, cwnd)

Whichever is smaller is the bottleneck. When a transfer is slow, you want to know which one it is. That is the subject of the next lesson.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does the window field in a TCP header tell the other side?

    The window field is the receiver's advertised free buffer space. It protects the receiver from being overwhelmed and is the basis of flow control.
  2. 2. A path has RTT = 80 ms and the receive window is 64 KB (65,535 bytes) with no scaling. About what is the highest throughput?

    Throughput is at most window / RTT = 65,535 x 8 bits / 0.08 s, which is about 6.55 Mbit/s, regardless of how fast the links are.
  3. 3. A connection is stuck with the receiver advertising a window of 0. What prevents a deadlock if the window update is lost?

    Window updates are not retransmitted. The sender probes with small segments (with exponential backoff) so it eventually learns about the reopened window.
  4. 4. What is the purpose of Nagle's algorithm?

    Nagle holds back small writes while data is in flight and sends them together once the ACK arrives, which cuts header overhead.
  5. 5. A client sends a request as two small writes and each request is delayed by about 40 ms. Which pair of behaviors is the likely cause?

    Nagle holds the second write until the first is acknowledged, and the receiver delays that ACK for up to about 40 ms. Writing the request in one piece or setting TCP_NODELAY avoids the stall.

Summary

  • Flow control keeps a fast sender from overrunning a slow receiver: the receiver advertises its free buffer space in the window field, and bytes in flight may not exceed it.
  • The window slides forward as ACKs arrive. A zero window stops the sender, and a persist timer sends probes so a lost window update cannot deadlock the connection.
  • Throughput is at most window / RTT. The 16-bit field limits the window to 65,535 bytes, so the window scale option (shift 0 to 14, negotiated in the SYN) raises the maximum to about 1 GiB.
  • Silly window syndrome is the flood of tiny segments caused by advertising tiny windows; Clark's rule and Nagle's algorithm prevent it on the receiver and sender.
  • Nagle plus delayed ACK can add about 40 ms to request-response traffic; write messages in one piece or use TCP_NODELAY.
  • The sender is limited by the smaller of the receive window and the congestion window.

Next, see how TCP discovers how much the network itself can carry in TCP congestion control.