Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 66 ยท Threads and Concurrency

Benefits and Pitfalls of Multithreading

See why multithreading improves responsiveness and speed, what limits the speedup (Amdahl's law), and how races, deadlocks and debugging problems appear, with simulations.

Multithreading is one of the most useful ideas in programming and also one of the easiest ways to create bugs that appear once a month and disappear when you add a print statement. This lesson balances both sides. First the benefits: responsiveness, resource sharing, economy and scalability on multicore CPUs. Then the limits: not every program speeds up, and the speedup has a mathematical ceiling. Finally the pitfalls, with a deterministic simulation of the most famous one, the race condition.

It also introduces two words used throughout the course: concurrency (several tasks making progress in overlapping time periods, possibly on one core) and parallelism (several tasks literally executing at the same instant on different cores).

Benefits

Responsiveness UI keeps working during slow tasks.
Resource sharing Threads share memory by default.
Economy Cheaper than processes.
Scalability Use all cores.
1 / 2

The four classic benefits

  1. Responsiveness. A GUI program can run a long calculation in a background thread while the interface thread keeps handling clicks. A single-threaded program would freeze.
  2. Resource sharing. Threads share their process's memory and files, so they exchange data with plain variables instead of IPC.
  3. Economy. Creating and switching threads costs much less than processes (context switching).
  4. Scalability. A single-threaded process uses one core. Threads of the same process can occupy all cores at once.

Threads also let you overlap waiting: while one thread waits for a network response, another can use the CPU. This is useful even on one core and is the reason web servers handle many clients at once.

Concurrency versus parallelism

  • Concurrency is about structure: dealing with many things at once. On one core, the OS interleaves threads quickly.
  • Parallelism is about execution: doing many things at once, which requires several cores.

You can have concurrency without parallelism (one core), and parallelism without concurrency in the programming sense (a vectorized loop on a GPU). Both kinds of work have different best-fit tools: I/O-bound work (waiting on network or disk) benefits from many threads even on one core, while CPU-bound work (heavy computation) benefits only if you have more cores.

The limit: Amdahl's law

Suppose a fraction s of your program is inherently serial (it cannot be split). With n cores, the best possible speedup is

speedup = 1 / (s + (1 - s) / n)

Even with unlimited cores the speedup cannot exceed 1 / s. If 10% of the work is serial, you can never go beyond 10 times faster, no matter how many threads you add. This is why more threads is not always better.

Python Amdahl's law table

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the 10% row: 8 cores give about 4.7x, 64 cores about 8.8x, and 1024 cores barely over 9.9x.

Pitfall 1: race conditions

A race condition occurs when the result depends on the exact order in which threads touch shared data. The textbook case is incrementing a shared counter. counter += 1 looks like one step, but the CPU does three: load the value, add one, store the result. If two threads interleave those steps, one update is lost.

The simulation below runs two "threads" as generators, each incrementing 5 times, and switches threads after every step. It is deterministic, so you can study the exact interleaving.

Python Lost updates

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The program prints 5, not 10. Both threads loaded the same old value, both added one, both stored the same result, so one increment vanished every time. The fix is to make the load-add-store sequence a critical section that only one thread can be in at a time, with a lock (mutex):

Python Fixed with a lock

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

(A real lock needs hardware support to make acquire atomic. The next chapter shows how.) Locks, semaphores and monitors are the subject of the Synchronization chapter of this course.

Pitfall 2: deadlock

If thread A holds lock 1 and waits for lock 2, while thread B holds lock 2 and waits for lock 1, both wait forever: a deadlock. Locks solve races but can create deadlocks, which get their own chapter.

Pitfall 3: other problems

  • Starvation: a thread never gets the lock or CPU time (starvation and aging).
  • Priority inversion: a low-priority thread holds a lock a high-priority thread needs.
  • Memory visibility: without proper synchronization one thread may not see another's writes promptly, because of CPU caches and compiler reordering.
  • Overhead: too many threads waste time in context switches and locks. More threads than cores rarely helps CPU-bound work.
  • Debugging: bugs depend on timing, so they appear rarely and vanish under a debugger (a "Heisenbug").
  • fork with threads: forking a multithreaded process copies only the calling thread, which leads to subtle bugs if another thread held a lock at that moment.

Guidelines for safe multithreading

  • Prefer no sharing: give each thread its own data and combine results at the end.
  • Prefer immutable data and message passing over shared mutable state.
  • Keep critical sections as short as possible and always take multiple locks in the same order.
  • Use higher-level tools (thread pools, concurrent queues, atomic counters) instead of raw threads where possible (thread pools).
  • Test under stress and use tools such as ThreadSanitizer to detect races.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the difference between concurrency and parallelism?

    Concurrent tasks can be interleaved on one core. Parallel tasks run simultaneously on different cores.
  2. 2. Under Amdahl's law, if 20% of a program is serial, what is the maximum possible speedup?

    The ceiling is 1 divided by the serial fraction, 1 / 0.2 = 5.
  3. 3. Why can two threads executing counter += 1 lose an update?

    If both load the same old value before either stores, one increment is overwritten.
  4. 4. Which technique prevents the race on a shared counter?

    A mutex makes the load-add-store sequence mutually exclusive.
  5. 5. Which workload benefits from many threads even on a single core?

    While one thread waits for I/O, others can use the CPU.

Summary

  • Multithreading gives responsiveness, sharing, economy and multicore scalability.
  • Speedup is limited by the serial fraction (Amdahl's law).
  • Shared mutable data causes race conditions, which locks fix; locks bring the risk of deadlock and contention.
  • Prefer less sharing, immutable data, short critical sections and consistent lock ordering.

Next: Thread Pools and Multitasking