Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 39 ยท Streaming

gRPC Flow Control, Backpressure and Streaming Patterns

Understand HTTP/2 flow control and backpressure in gRPC streams, bounded queues and drop policies, batching, and resumable streams. Includes runnable Go stand-ins.

A stream is a promise: messages keep coming. What happens when the sender is faster than the receiver? If nothing limits it, memory fills up until the process dies. gRPC relies on HTTP/2 flow control to push back on fast senders, and your code decides what happens above that. This final streaming lesson explains the mechanism, then collects the practical patterns: bounded queues, drop policies, batching and resumable streams.

Real gRPC code stays read-only on this site. The Go stand-ins are runnable and deterministic.

Flow control in HTTP/2

Every HTTP/2 stream, and the connection as a whole, has a window: the number of bytes the sender may transmit before it must wait for a WINDOW_UPDATE frame from the receiver. The receiver grants more credit only as its application reads data. The default initial window is small (64 KB in the protocol; gRPC implementations adapt it dynamically using bandwidth-delay estimates).

Backpressure in four steps

Sender writes Uses up its window
Window is empty Send blocks
Receiver reads Frees buffer space
WINDOW_UPDATE Sender continues

A slow reader automatically slows the writer.

1 / 2

The consequence for code: stream.Send can block. A server that sends faster than a client reads will simply stop in Send until the client catches up. That is backpressure working as designed. It also means a stuck client can hold a server goroutine and its memory forever, unless you use deadlines, keepalives or cancellation (see deadlines and cancellation).

Stand-in 1: credit-based flow control

This runnable program models the window mechanism: a sender may have at most 4 unacknowledged messages in flight. The receiver acknowledges as it processes. The program tracks the maximum in flight and shows that it never exceeds the window.

Go Stand-in: a send window

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Your queues need bounds too

Flow control protects the wire. Your own code can still accumulate unbounded data, for example a hub that buffers messages for each subscriber (see the chat hub in bidirectional streaming). Always use bounded queues and choose a policy for when they fill:

Policy Behavior Suits
Block The producer waits Pipelines where every item matters and the producer can wait
Drop newest Refuse the incoming item Rate-limiting bursts
Drop oldest Evict stale items to make room Live prices, positions, sensor data: only the latest matters
Disconnect Close the slow consumer's stream Chat hubs, where one slow client must not hurt others
Coalesce Merge items by key State updates: keep only the latest value per key

Stand-in 2: drop-oldest and coalescing queues

Go Stand-in: bounded queue policies

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

For a stock ticker, coalescing means a slow client sees the latest price per symbol instead of a backlog of stale ones.

Batching

Sending one tiny message per event wastes framing and syscalls. Batch several events into one message:

PROTO
message PriceBatch {
  repeated PriceUpdate updates = 1;
}

Flush the batch when it reaches a size (say 100 events) or an age (say 50 ms), whichever comes first. This trades a little latency for much higher throughput. Measure before tuning; the right numbers depend on message size and network.

Resumable streams

Streams break: deploys, network blips, load balancer idle timeouts. Design streams so clients can resume without losing or duplicating data:

  1. Give every event a monotonically increasing sequence number or id.
  2. The client remembers the last id it processed.
  3. On reconnect, the request carries resume_after = lastID.
  4. The server replays from that point (it needs a bounded history buffer, or a durable log).
  5. Processing is idempotent on the client, so a replayed event does no harm.
PROTO
message SubscribeRequest {
  repeated string topics = 1;
  int64 resume_after = 2; // 0 = start from now
}

Stand-in 3: replay after a reconnect

Go Stand-in: resume from the last id

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Load balancing and streams

A stream stays on one backend for its entire life. A fleet of long-lived streams can therefore become unbalanced, especially after a deploy, when the first backend to restart collects everyone. Mitigations: set a maximum connection age on servers (so clients reconnect and rebalance), spread reconnects with random jitter, and let clients re-resolve names. The load balancing chapter on the track page covers this in depth.

Keepalive settings (read-only)

Go
// server
grpc.KeepaliveParams(keepalive.ServerParameters{
	MaxConnectionAge:      30 * time.Minute, // force periodic reconnects
	MaxConnectionAgeGrace: 30 * time.Second,
	Time:                  2 * time.Minute,  // ping idle clients
	Timeout:               20 * time.Second,
})

Do not set pings too aggressively; servers and proxies punish clients that ping too often (the GOAWAY "too_many_pings" error).

Choosing a call type: a recap

Need Call type
Simple question and answer Unary
Big result, feed, progress Server streaming
Upload, batch ingestion Client streaming
Real-time two-way traffic Bidirectional streaming

Start with unary. Move to streaming when you can name the benefit: lower latency, bounded memory, or less overhead.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why can stream.Send block?

    Window-based flow control stops senders from overwhelming slow receivers.
  2. 2. For a live stock ticker, which queue policy fits a slow client best?

    Only the latest price matters, so stale values may be dropped.
  3. 3. What makes a stream resumable?

    The client reports the last processed id, the server replays after it, and duplicates are harmless.
  4. 4. Why set a maximum connection age on streaming servers?

    A stream sticks to one backend, so periodic reconnects spread load again.
  5. 5. Why batch events into one message?

    Batching amortizes framing costs, at the price of slightly higher latency.

Summary

  • HTTP/2 windows create backpressure: slow readers slow fast writers, and Send can block.
  • Bound every queue and choose a policy: block, drop newest, drop oldest, disconnect or coalesce.
  • Batch small events; flush by size or age.
  • Make streams resumable with ids, a resume position and idempotent handling.
  • Mind load balancing: long-lived streams need connection-age limits and jittered reconnects.

Next: back to the gRPC and Protocol Buffers track for the chapters on error handling, security, load balancing and testing.