Skip to content
Tutorials Center
My learning

Topics

Lesson 71 of 147 ยท Concurrency

Concurrency Patterns - Pipelines, Worker Pools, Fan-out

Learn Go concurrency patterns with runnable code - generators, pipelines, worker pools, fan-out and fan-in, semaphores, rate limiting, an errgroup-style helper and bounded parallelism.

You now know the pieces: goroutines, channels, select, the sync package and context. Real programs combine them into a handful of recurring shapes. Learning these shapes is like learning data structures: once you recognize the problem, the solution comes quickly and you know where the traps are.

This lesson builds the classic patterns step by step: the generator, the pipeline, the worker pool, fan-out and fan-in, the semaphore, rate limiting, an errgroup-style helper (written by hand, because the standard library has no errgroup) and bounded parallelism. Every example is deterministic, so the output you see is the output you get. Where goroutines finish in random order, the code stores results by index or sorts them, and the text points that out.

The generator

A generator is a function that returns a channel and produces values on it from a goroutine. The caller simply ranges over the result. The generator owns the channel, so it closes the channel when done, and it takes a context so the consumer can walk away without leaking the goroutine:

Go A generator with cancellation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The output is 0 1 1 2 3 5 8 13 21 34. The sequence is infinite, yet the program ends cleanly because cancel() stops the producer. Since Go 1.23, a lot of what used to need a generator goroutine can be written as an iterator instead, which needs no goroutine at all. Reach for channels when the producer truly runs concurrently with the consumer.

Pipelines

A pipeline is a chain of stages. Each stage receives values from an inbound channel, does some work, and sends results on an outbound channel. Stages run concurrently, so while one handles item 3, the previous stage is already working on item 4.

The shape of every stage is the same: take <-chan T, return <-chan U, close the output when the input ends.

Go A three-stage pipeline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The output is GO!, IS!, FUN!, in order. Because each stage handles values one at a time in sequence, a pipeline of single-goroutine stages keeps the order of the input. Closing flows downstream: when emit finishes it closes its channel, upper's range ends and it closes its own channel, and so on until main's loop ends.

The stages above repeat the same loop. With generics you can write the stage once:

Go A generic pipeline stage

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

You get #4 #16 #36 #64 #100. Here three different element types flow through the same helper functions (see Generics). In real code, give each stage a ctx parameter and make sends select against ctx.Done(), like the generator above, so an early exit can never strand a goroutine.

The worker pool

A pipeline stage can become a bottleneck if its work is slow. A worker pool starts a fixed number of goroutines that all read from one shared jobs channel. Whichever worker is free takes the next job, so the load balances itself. Results go to a results channel:

Go A worker pool with a jobs and a results channel

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The output is [1 4 9 16 25 36 49 64]. The workers finish jobs in an unpredictable order, so the code stores each result at the slot given by its job number. Three details make this pattern correct:

  1. The producer closes jobs, which tells the workers to stop.
  2. A separate goroutine waits for the workers and then closes results, because no single worker knows it is the last.
  3. main ranges over results, so it ends exactly when all results have arrived. If main waited on the WaitGroup itself before reading results, the workers would block sending and nothing would move.

How many workers? For CPU-bound work, about the number of cores (runtime.NumCPU()). For work that mostly waits on a network or disk, many more, limited by what the other side can handle: database connections, API rate limits, open files.

Fan-out and fan-in

The worker pool is a combination of two ideas:

  • Fan-out: several goroutines read from the same channel, spreading the work.
  • Fan-in: several channels are merged into one, collecting the results (fan-in with merge).

A pool uses a shared input channel for fan-out and a shared output channel for fan-in. When each worker should have its own output (for example to keep results grouped), you fan-out to separate channels and merge them afterwards:

Go Fan-out to workers, then fan-in with merge

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The sorted result is [1 4 9 16 25 36 49 64 81]. Without the sort the order would vary from run to run, because three goroutines compete for inputs. Whenever you fan out, the output order is lost. If order matters, tag each item with its index (as in the worker pool above) and reassemble the results at the end.

Bounded parallelism with a semaphore

A worker pool fixes the number of goroutines up front. Sometimes it is easier to start one goroutine per task but allow only N to run at the same time. A buffered channel makes a perfect semaphore: sending acquires a slot (blocking when all N are taken), receiving releases it.

Go At most three tasks run at once

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Both lines are deterministic: peak concurrency never above limit: true and everything finished, running now: 0. The exact peak (1, 2 or 3) depends on scheduling, so the program prints only the safe comparison. The CompareAndSwap loop to record the peak is the technique from the atomics lesson.

Ten goroutines are created, but at most three work at a time; the rest wait on the semaphore. This style is good when the number of tasks is moderate. For millions of tasks, a worker pool is lighter, because it does not create a goroutine per task. Since Go 1.25, wg.Go is the neat way to launch each one, and there is also golang.org/x/sync/semaphore in the extended libraries for weighted semaphores.

Rate limiting

To avoid overwhelming a service (or to respect an API quota) you may need to limit how often something happens. The simplest limiter is a time.Ticker: wait for the next tick before each operation.

Go Limiting the pace with a ticker

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Five requests, one per 25 ms tick, take at least 100 ms in total. The program prints the five request lines and then took at least 100ms: true (it prints a comparison instead of the exact time, which would differ on every run).

To allow short bursts while keeping the average rate, fill a buffered channel with tokens and refill it on a ticker. A burst of up to N requests goes through instantly, then the limiter slows down to the refill rate:

Go A token bucket that allows a burst

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Requests 1 to 3 use the stored tokens immediately; requests 4 and 5 each wait for a refill, roughly 30 ms apart, so the total is at least about 60 ms and the final comparison is true. For production, the extended library golang.org/x/time/rate offers a ready-made token-bucket limiter with context support. The stdlib-only versions above show how it works.

An errgroup-style helper

Often a group of goroutines should stop as soon as one fails, and the caller wants the first error. The extended library has golang.org/x/sync/errgroup for this, but it only takes about twenty lines to build one yourself with a WaitGroup, sync.Once and a context:

Go A hand-built errgroup

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The failing task returns disk full after 20 ms, which cancels the shared context. The slow task, which would have waited five seconds, stops at once and prints slow task stopped early: disk full. Wait then returns the first error: group error: disk full. Because sync.Once guards the assignment, the later context canceled error from the slow task is not recorded.

The pieces fit together neatly: WaitGroup counts tasks, Once keeps the first error, and a cancelable context is the shared stop signal. The real errgroup.Group adds SetLimit for bounded parallelism and TryGo.

Bounded parallelism over a slice

A very common job is "process all of these items, N at a time, and collect the results in the original order". Combining the semaphore with indexed results solves it, with no shared mutable state except distinct slice slots:

Go Parallel map with a limit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The output keeps the input order even though the work runs two or three at a time: [5 4 5 5 7 4], then [ALPHA BETA GAMMA DELTA EPSILON ZETA]. Acquiring the slot in the loop (not inside the goroutine) means at most limit goroutines exist at any moment, which keeps the memory use bounded for huge slices too.

Pitfalls to remember

  • Every goroutine needs an exit. Before writing go, know what makes it stop: input channel closed, context cancelled, or a task that simply ends.
  • The sender closes. Producers close their output; consumers never do. With several producers, use a WaitGroup and a closer goroutine.
  • Fan-out loses order. Index your results if order matters.
  • Unbounded concurrency is a bug waiting to happen. One goroutine per item is fine for ten items, but for a million it can exhaust memory, file descriptors or a remote service. Always bound it.
  • Do not forget errors. A goroutine that fails silently is worse than one that crashes. Collect errors with a group or send them on a channel.
  • Keep it simple. If a plain loop is fast enough, use the loop. Concurrency adds complexity, and every shared variable needs a clear owner. Measure before you parallelize, and test with the race detector, as described in Data Races and the Memory Model.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In the worker-pool pattern, who should close the jobs channel?

    The sender owns a channel and closes it. Closing jobs ends the workers' range loops. Workers only receive from it.
  2. 2. Why is results closed in a separate goroutine after wg.Wait()?

    Several workers send on results, so none can close it safely. A helper goroutine waits for all of them and closes it once, which lets the consumer's range loop end.
  3. 3. What happens to result order when several goroutines read from one input channel and write to one output channel?

    Workers finish at different times, so results arrive in arbitrary order. Keep an index with each item and reassemble if order matters.
  4. 4. How do you build a semaphore that allows at most 4 concurrent tasks?

    A send blocks once 4 values are buffered, so a fifth task waits until some task receives (releases) a slot.
  5. 5. In a hand-built errgroup, why is sync.Once used when recording the error?

    Several tasks may fail concurrently. Once guarantees that exactly one error is stored, and a race on the err field is impossible.

Summary

  • A generator returns a channel fed by a goroutine; it owns and closes the channel and watches ctx.Done() so consumers can stop early.
  • A pipeline chains stages shaped func(<-chan T) <-chan U; closing flows downstream, and generic helpers like Map and Filter avoid repetition.
  • A worker pool starts N goroutines reading a shared jobs channel; the producer closes jobs, a helper goroutine closes results after wg.Wait().
  • Fan-out spreads work over goroutines, fan-in merges outputs; both lose input order, so index results when order matters.
  • A buffered channel is a semaphore for bounded parallelism; a ticker or token bucket gives rate limiting.
  • An errgroup is a WaitGroup + sync.Once + a cancelable context: first error wins, the others are told to stop.
  • Always bound concurrency, give every goroutine an exit, let senders close channels, and keep things simple when a loop would do.

Next: Data Races and the Memory Model explains exactly what goes wrong when goroutines share data without synchronization.