Skip to content
Tutorials Center
My learning

Topics

Lesson 74 of 147 ยท Concurrency

Testing Concurrent Code

Learn to test goroutines, channels and timers in Go without flaky sleeps - deterministic waits, testing/synctest with a fake clock, goroutine leak checks and running the race detector in CI.

Concurrent code is the hardest code to test, for a simple reason: a test that passes ninety-nine times and fails on the hundredth is worse than no test. It teaches the team to re-run the build until it goes green, and then a real bug slips through. The usual culprits are time.Sleep calls that guess how long something takes, assertions that run before a goroutine has finished, and orderings that depend on the scheduler.

This lesson shows how to write tests that give the same answer every time. You will wait on real events instead of sleeping, put a timeout only as a safety net, use the testing/synctest package (added in Go 1.25) to run timers on a fake clock, detect leaked goroutines, and let the race detector watch your tests in CI. Basic testing syntax is in the testing lesson and the machinery of goroutines and channels is in goroutines and channels.

This site cannot run go test, so the runnable examples build their own tiny test runner in main. They use the real testing and testing/synctest packages. In your own projects you put the TestXxx functions in a _test.go file and run go test, exactly as the read-only snippets show.

Why sleeping in tests fails

Here is a test written the tempting way. It is read-only, because the point is that it is unreliable:

Go
func TestWorkerFlaky(t *testing.T) {
	var done bool
	go func() {
		doWork()
		done = true
	}()

	time.Sleep(100 * time.Millisecond) // "should be enough"
	if !done {
		t.Fatal("work not finished")
	}
}

This test has three problems at once:

  • Too short on a slow machine. A busy CI server may need more than 100 ms, and the test fails for no real reason.
  • Too long everywhere else. Every run waits the full 100 ms even when the work took a microsecond. Multiply that by hundreds of tests.
  • It is a data race. done is read and written by two goroutines with nothing ordering them (see data races). sleep creates no happens-before relationship.

The fix is always the same: wait for the event you care about, not for a guess about time.

Technique 1: wait on a channel or WaitGroup

If the code under test finishes by sending a result, closing a channel or calling wg.Done(), the test can simply wait for that. A WaitGroup or channel receive orders everything the goroutine did before the test continues, so reading the results afterwards is race-free.

For channel-based code, add a small helper with a timeout that only exists to fail fast. A passing test never waits for it, and a broken one fails in seconds instead of hanging the whole suite:

Go Testing a pipeline stage without sleeping

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The test never sleeps. The unbuffered send waits for Square to be ready, the receive waits for the answer, and closing in makes Square close out. The 2-second timeout is never reached when the code is correct.

Some rules of thumb for this style of test:

  • Make the timeout long (seconds). It is a failure detector, not a performance check, and a long value avoids flakiness on slow machines.
  • Prefer t.Fatal over t.Error after a timeout, since later checks usually make no sense.
  • Do not call t.Fatal or t.FailNow from a goroutine other than the one running the test. They must run on the test goroutine. Send errors back over a channel or use t.Error (which is safe from any goroutine) and return.
  • Design for testability: functions that accept a channel, a context.Context or a callback are much easier to test than ones that hide a goroutine and give no way to know when it is done.

Technique 2: a fake clock with testing/synctest

Some code is about time itself: retries with backoff, heartbeats, cache expiry, timeouts. Testing it with real time makes the suite slow (a test about a 30-second timeout takes 30 seconds) and flaky (the margin for error is a few milliseconds).

Go 1.25 added the testing/synctest package for exactly this. synctest.Test(t, func(t *testing.T) { ... }) runs the function in an isolated bubble. Inside the bubble:

  • The time package uses a fake clock. It starts at midnight UTC on January 1, 2000, and it only moves forward when every goroutine in the bubble is durably blocked, for example on time.Sleep, a timer, a channel operation or a sync.WaitGroup.Wait on a bubble-created group. Then the clock jumps straight to the next timer. A ten-hour sleep takes no real time at all.
  • synctest.Wait() blocks until all other goroutines in the bubble are durably blocked. After it returns you know everyone else has finished reacting to what just happened, so you can inspect shared state without locks or sleeps.

(Go 1.24 had an experimental synctest.Run function that was replaced by synctest.Test. Use Test.)

Our program needs a real *testing.T for synctest.Test, so the example below uses testing.Main to run three test functions the way go test would, in verbose mode. This is only a harness for the lesson; in a real project you write the same TestXxx functions in _test.go.

Go Backoff and heartbeats on a fake clock

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

All three pass in a few milliseconds of real time, although the code under test "waited" 3 seconds, 3 minutes and 5.5 seconds. The assertion got != 3*time.Second compares with ==-style exactness, which would be impossible with a real clock. Because the clock only moves when everything is blocked, the order of events is deterministic: the tick at 5 seconds is always delivered before the sleep ends at 5.5 seconds.

Using synctest.Wait

synctest.Wait() is the tool for the question "has the goroutine I started finished reacting yet?". Instead of sleeping, you call Wait and then look at the state. The state can be read without a mutex because Wait creates the necessary happens-before edge. The example below uses the same small harness as before:

Go synctest.Wait instead of Sleep

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The second test checks a context deadline to the millisecond: one millisecond before the deadline there is no error, and exactly at the deadline context.DeadlineExceeded appears. Doing that with a real clock would be a flaky guess.

Limits to know about:

  • Only goroutines started inside the bubble belong to it. Blocking on something outside the bubble (real network I/O, a channel created outside) does not count as durably blocked, so the clock will not advance past it.
  • Channels, timers and tickers created inside the bubble are bound to it. Using them from outside is a panic.
  • Real file or network operations do not mix with fake time. Fake the dependency with an interface or use net.Pipe, which works in a bubble.

Technique 3: catching goroutine leaks

A leaked goroutine is one that is blocked forever, such as a worker waiting on a channel nobody will ever send to. Each leak keeps its stack and everything it references alive. synctest finds leaks for free: when the function passed to synctest.Test returns while goroutines in the bubble are still blocked and can never wake up, the test fails with a deadlock panic:

Go synctest reports a leaked goroutine (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

The test is reported as failed, and the program panics with:

TEXT
panic: deadlock: main bubble goroutine has exited but blocked goroutines remain [recovered, repanicked]

followed by a stack trace that includes a goroutine marked goroutine N [chan receive (durable), synctest bubble 1] that points straight at the line of the for range jobs loop. The fix is the missing close(jobs), or a context the worker listens to.

Outside a bubble, a simple guard is to compare runtime.NumGoroutine() before and after. Goroutines need a moment to exit, so poll for a short time instead of checking once:

Go A hand-made leak check

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

This check polls for up to one second, which is why the leaky case takes a moment to report false. It is crude: it cannot say which goroutine leaked, and other tests running in parallel would disturb the count. Use it only as a quick guard. The third-party go.uber.org/goleak package gives a more precise version, and a synctest bubble is the most precise tool when your code is compatible with it.

Technique 4: let the race detector watch your tests

Passing tests do not prove that there are no data races. A race often does no visible damage until production. The race detector turns most races into test failures:

Terminal
# Run every test with the race detector
go test -race ./...

# Repeat tests to give rare interleavings a chance, in random order
go test -race -count=20 -shuffle=on ./...

# Fail a hung test after 30 seconds instead of the default 10 minutes
go test -race -timeout=30s ./...

Make this the default in CI. A typical GitHub Actions job (read-only, adjust it to your project):

YAML
name: test
on: [push, pull_request]
jobs:
  go-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version-file: go.mod
      - run: go vet ./...
      - run: go test -race -count=3 -timeout=5m ./...

The detector only sees races that happen during the run, so your tests must actually exercise the concurrent paths: start several goroutines, call the same method from many of them, and use t.Parallel() for tests that are safe to overlap. A test that calls a thread-safe type from just one goroutine will pass -race even if the type is broken. More on the detector itself is in data races.

A checklist for stable concurrent tests

  • Never sleep to wait. Wait on a channel, a WaitGroup, a context, or synctest.Wait().
  • Use timeouts only as safety nets. Make them generous, and set -timeout on the whole run.
  • Use a fake clock for time logic. testing/synctest, or an injected now func() time.Time as shown in time.
  • Shut down what you start. Close channels and cancel contexts so the goroutines exit, and clean up with t.Cleanup.
  • Keep goroutines out of assertions. Collect results over channels, and call t.Fatal only from the test goroutine.
  • Run with -race and repeat. Use -count and -shuffle=on locally when you suspect flakiness.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why is time.Sleep(100 * time.Millisecond) a bad way to wait for a goroutine in a test?

    A sleep is a guess. Too short and the test fails on a busy machine; too long and every run wastes time. It also creates no happens-before relationship, so reading shared data afterwards can still be a data race.
  2. 2. What does synctest.Test do to the time package inside its function?

    Inside a synctest bubble the clock is virtual. It only advances when every goroutine in the bubble is durably blocked, and then it jumps straight to the next timer, so a 10-hour sleep returns instantly.
  3. 3. What is synctest.Wait() for?

    After Wait returns, every other goroutine in the bubble has finished reacting to what happened and is waiting again. You can then read shared state without a lock or a guess.
  4. 4. Where must t.Fatal be called?

    t.Fatal calls runtime.Goexit, which stops only the calling goroutine. From another goroutine it would not stop the test. Use t.Error there, or send the error back to the test goroutine.
  5. 5. A go test -race ./... run reports no races. What can you conclude?

    The race detector reports races it observes while the program runs. A clean run is good evidence, but it is not a proof; untested paths and rare interleavings can still hide races.

Summary

  • Flaky concurrent tests usually come from time.Sleep guesses and from assertions that run before the goroutine has finished. Wait for events instead: channel receives, WaitGroup.Wait, context cancellation.
  • Use generous timeouts as failure detectors only, and set go test -timeout for the whole run.
  • testing/synctest (Go 1.25) runs code in a bubble with a fake clock that advances when all goroutines are blocked, so tests of backoff, tickers and deadlines are exact and instant. synctest.Wait() replaces sleeps when you need to inspect state.
  • synctest.Test fails a test that leaves goroutines blocked forever; elsewhere, compare runtime.NumGoroutine() with polling or use a leak-detection library.
  • Call t.Fatal only from the test goroutine; use t.Error or channels from other goroutines.
  • Run go test -race -count=N -shuffle=on ./... locally and go test -race ./... in CI, and make sure tests exercise concurrent paths.

Next: fmt in Depth: Formatting and Scanning goes through every printing verb and flag.