Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 40 ยท Synchronization and Queues

Semaphores and Rate Limiting in asyncio

Learn asyncio.Semaphore and BoundedSemaphore to cap how many tasks run at once, build a concurrency limiter for downloads, and add simple rate limiting.

asyncio.gather with a thousand coroutines starts a thousand jobs at once. That is wonderful until each job opens a network connection, and the remote server (or your own operating system) refuses more than a few dozen. You need a way to say "run these jobs concurrently, but never more than N at a time". That is exactly what a semaphore does.

What a semaphore is

An asyncio.Semaphore(n) holds a counter that starts at n. Each async with semaphore: takes one permit (decrements the counter). When the counter reaches zero, the next task to ask waits until some other task releases a permit. A Lock is simply a semaphore with n = 1.

Semaphore(3): three permits

3 permits Counter starts at 3
Task enters Takes a permit
Counter hits 0 Next task must wait
Task leaves Permit goes back

At most 3 tasks inside at any moment

1 / 2

Limiting concurrent jobs

Twelve jobs each take 0.1 seconds. With a limit of 3, at most three run together, so they finish in about four waves. The counters prove the limit is respected:

Python At most three at a time

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All twelve tasks exist from the start, but only three at a time are inside the async with block. peak never exceeds 3. Note that the semaphore must be shared: create it once and pass it to every task. Creating a new semaphore inside each task limits nothing.

A reusable pattern: bounded gather

You will write this helper often, so keep it handy. It runs any list of coroutine functions with a concurrency cap and returns results in order:

Python A bounded_gather helper

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the output: a job only prints start once it holds a permit, so at most two jobs are between their start and end lines at any moment. Results are still returned in order.

BoundedSemaphore: catch release bugs

A plain Semaphore lets you call release() more times than acquire(), silently raising its limit above the original value, which is almost always a bug. asyncio.BoundedSemaphore raises ValueError in that case, so mistakes show up immediately:

Python BoundedSemaphore protects you

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

(_value is a private attribute shown only for this demonstration; never rely on it in real code.) When you use async with, acquire and release are always paired, so the problem only appears with manual acquire() and release() calls.

Rate limiting: N requests per time window

A semaphore limits how many at once. A rate limit is different: "at most 5 requests per second". A simple approach: hold the permit for a fixed minimum time so that permits free up at a steady rate.

Python A tiny rate limiter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Nine calls are released in batches of three, one batch per 0.2-second window. The permit is returned by a timer (call_later), not by the caller, which is why the limiter works even if calls finish instantly. Production code usually uses a library with a token bucket or leaky bucket algorithm (such as aiolimiter), but the idea is the same.

Semaphores with TaskGroup

TaskGroup has no built-in limit, so combine them: create the group, create all tasks, and let the semaphore inside each task do the throttling. Be careful with memory when you have millions of items. Creating a task per item still allocates a million task objects up front. For huge workloads use a queue with a fixed pool of workers instead, which creates only N tasks.

Common mistakes

  • Creating the semaphore inside the task function, so each call gets its own full set of permits.
  • Holding the permit while doing unrelated slow work. Keep the async with block as narrow as the resource being protected.
  • Using a semaphore for CPU limits. It limits concurrency on the loop, not CPU use.
  • Forgetting that semaphores are not thread-safe, like all asyncio primitives.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Semaphore(3): how many tasks can be inside the async with block at once?

    The semaphore starts with 3 permits; a fourth task waits until one is released.
  2. 2. What is a Lock in terms of semaphores?

    A lock allows exactly one holder, so it behaves like a semaphore with one permit.
  3. 3. What does BoundedSemaphore add?

    It catches over-release bugs that would silently raise the limit.
  4. 4. Why does creating a new Semaphore inside each task fail to limit concurrency?

    The limit only works when all tasks share one semaphore object.

Summary

  • Semaphore(n) lets at most n tasks into a block; others wait without blocking the loop.
  • Share one semaphore among all tasks; use async with so permits are always returned.
  • BoundedSemaphore flags extra releases.
  • Rate limiting (N per second) differs from concurrency limiting: return permits on a timer.
  • For huge job counts, prefer a fixed worker pool over one task per item.

Next: Queues and the producer-consumer pattern