Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 40 ยท Testing and Debugging

Common asyncio Mistakes and How to Avoid Them

A field guide to asyncio pitfalls: missing await, blocking the loop, lost tasks, check-then-act races across awaits, sequential awaits, swallowed cancellation, and task-local state.

asyncio rewards a small mental model and punishes small misunderstandings. The mistakes below account for most of the "my async code is slow / hangs / loses data" questions that developers ask. For each, you get the symptom, the cause and the fix, and for the subtle ones a runnable demonstration.

Keep this page as a checklist for code reviews of async code.

1. Forgetting await

Symptom: a function appears to do nothing; a warning says coroutine ... was never awaited. Cause: calling an async def function only creates a coroutine object. Fix: await it, or schedule it with asyncio.create_task(). Linters catch this, and debug mode reports it.

2. Blocking the event loop

Symptom: everything stutters; timeouts fire late; heartbeats stop. Cause: time.sleep, requests, synchronous database drivers, big file reads or heavy computation inside a coroutine. Fix: use async libraries, or offload with asyncio.to_thread and process pools (running blocking code).

3. Awaiting in a loop when you wanted concurrency

Symptom: fetching 50 URLs takes 50 times as long as one. Cause: for url in urls: results.append(await fetch(url)) waits for each call before starting the next. Fix: start them together with gather or a TaskGroup, and bound the number with a semaphore.

Python Sequential awaits versus concurrent tasks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The sequential version needs several times as many turns (five calls of ten turns each, one after another) while the concurrent version overlaps all five calls.

4. Check-then-act races across an await

Symptom: counters are wrong, items are processed twice, limits are exceeded: but only sometimes. Cause: asyncio has no preemption, so code between awaits is atomic, but any await is a point where another task may run and change shared state. A "check, await, then act" sequence is a race. Fix: make the check and the update happen without an await between them, or protect the whole sequence with an asyncio.Lock.

Python A race with a missing lock

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Without the lock both tasks passed the check (100 is at least 80) before either subtracted, so the account went negative. The lock makes the check and the update one indivisible step relative to other tasks. See the synchronization chapter for Lock, Event, Condition and Semaphore.

5. Losing tasks and their exceptions

Symptom: a background job silently stops; Task exception was never retrieved in the logs; a task is destroyed while pending. Cause: the event loop keeps only a weak reference to tasks, and nobody awaited the result. Fix: store tasks in a set (remove them in a done callback) or use a TaskGroup, and always log failures, as shown in debugging.

6. Swallowing CancelledError

Symptom: tasks do not stop on shutdown; timeouts "don't work"; the program hangs at exit. Cause: except BaseException:, a bare except:, or catching CancelledError without re-raising. Fix: catch Exception for ordinary errors, use finally for cleanup, and re-raise CancelledError. Details in cancelling tasks.

7. Using thread primitives in coroutines

Symptom: the whole program deadlocks. Cause: threading.Lock, queue.Queue.get() or threading.Event.wait() called inside a coroutine block the thread, so the task that would release them can never run. Fix: use the asyncio versions (asyncio.Lock, asyncio.Queue, asyncio.Event). They are not thread-safe, so across threads use call_soon_threadsafe or run_coroutine_threadsafe.

8. Unbounded concurrency and queues

Symptom: "Too many open files", memory growth, 429 responses, or a banned IP. Cause: gather over 100,000 items, or asyncio.Queue() with no maxsize. Fix: semaphores, worker pools, rate limiters and bounded queues (concurrency limits, backpressure).

9. Calling asyncio.run() from a running loop

Symptom: RuntimeError: asyncio.run() cannot be called from a running event loop. Cause: asyncio.run creates a new loop, so it belongs at the top of the program only; calling it from async code (or from Jupyter, which already runs a loop) fails. Fix: inside async code just await the coroutine. In a notebook use top-level await. If sync code must call async code from a worker thread, use asyncio.run_coroutine_threadsafe(coro, loop).

10. Passing a coroutine object where a callable is needed

Symptom: RuntimeError: cannot reuse already awaited coroutine in a retry loop. Cause: a coroutine object can be awaited once. Retry helpers need a factory: lambda: fetch(url), not fetch(url) (retries).

11. Forgetting timeouts

Symptom: a request hangs for minutes; a worker is stuck forever on a socket or lock. Cause: most awaits wait indefinitely by default. Fix: wrap external waits in asyncio.timeout() (timeouts).

12. Sharing "global" state between tasks

Symptom: logs and request data from different requests get mixed up. Cause: a module-level variable such as current_user is shared by every task. Fix: use context variables. Each task gets its own copy of the context at creation time, so a value set in one task is invisible to the others, and it still follows the code across awaits.

Python contextvars give each task its own value

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Every task reads its own request_id back correctly, while the shared dictionary ends up holding whichever request wrote last. Context variables are how logging libraries and tracing tools attach a request ID to every log line without passing it through every function.

A quick review checklist

  • Does every call to an async def have await, or a task around it?
  • Is anything inside a coroutine blocking (sleep, requests, disk, CPU)?
  • Is there a timeout on every external wait?
  • Is concurrency bounded, and are queues bounded?
  • Are background tasks referenced and their failures logged?
  • Is CancelledError re-raised, and is cleanup in finally?
  • Is shared state touched across an await without a lock?

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is "for url in urls: results.append(await fetch(url))" slow?

    Awaiting inside the loop serialises the calls. Use gather or a TaskGroup to start them concurrently.
  2. 2. Where can another task change shared state in the middle of your coroutine?

    asyncio is cooperative; tasks switch only at awaits, so code between awaits is atomic, but a check-then-await-then-act sequence is not.
  3. 3. What is wrong with calling threading.Lock().acquire() inside a coroutine?

    A blocking acquire holds the single thread, so the task that would release the lock cannot run. Use asyncio.Lock.
  4. 4. What is the best tool for per-request data such as a request ID that must not leak between concurrent tasks?

    Each task runs in a copy of the context, so a ContextVar value set in one task is invisible to others.

Summary

  • The big ones: missing await, blocking calls, serial awaits, check-then-act races, lost tasks, swallowed cancellation, unbounded concurrency and missing timeouts.
  • asyncio switches tasks only at await; protect multi-step updates that span an await with a lock.
  • Use asyncio primitives, never thread primitives, in coroutines; use ContextVar for task-local state.
  • Review async code against the checklist above.

Next: Project: a concurrent web crawler