Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 40 ยท Tasks and Gathering

Handling Exceptions in asyncio Tasks

Learn how exceptions behave in asyncio tasks, the "exception was never retrieved" warning, try/except around await, retries with backoff and a safe fire-and-forget helper.

In ordinary Python an exception travels up the call stack until someone catches it. Tasks change that picture: a task runs on its own, so when it fails there may be no caller on the stack. Where does the exception go? This lesson answers that and shows patterns that keep errors visible instead of silently lost.

Rule 1: the exception waits inside the task

When a coroutine running as a task raises, the task does not crash the program. It finishes in a "failed" state and stores the exception. The exception reaches you only when you ask for it, by awaiting the task or calling task.result() or task.exception().

Python Exceptions are stored in the task

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Nothing crashed while the task was failing; the program kept running until we looked. That is convenient, but it is also dangerous.

On this site's editor you may also see a traceback logged in the error output at the moment the task fails. That is the editor's own loop reporting the failure; in a normal Python program nothing is printed until you retrieve the exception or the task is garbage collected.

Rule 2: never retrieved means never seen

If a task fails and nobody ever awaits it or reads its exception, asyncio logs Task exception was never retrieved when the task object is garbage collected, and prints the traceback to the log. In a long-running server that message may be your only clue, and it can appear long after the real failure.

Python
async def main():
    asyncio.create_task(risky())      # fire and forget: BUG-prone
    await asyncio.sleep(1)

On your computer this logs the traceback of risky with the message above. (The log is read-only here because it depends on garbage collection timing.) The fix is to make sure every task is awaited or has its result observed, using a TaskGroup or a done callback.

Rule 3: catch exceptions where you await

Wrap the await in try/except just like a normal call:

Python try/except around await

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For several tasks, gather with return_exceptions=True and TaskGroup with except* are the right tools.

Rule 4: observe fire-and-forget tasks with a callback

Sometimes you genuinely want to start a background task and move on. Add a done callback that logs failures, and keep a reference so the task is not collected:

Python A safe fire-and-forget helper

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The callback runs when the task finishes, so a failure is reported immediately and consistently. The callback itself must be a plain function (it cannot await) and must not raise.

Retries with exponential backoff

A common reason to catch task exceptions is retrying flaky operations. Wait longer after each failure so a struggling service is not hammered:

Python Retry with backoff

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Only retry errors that are worth retrying (timeouts, connection resets), never programming errors like TypeError. Add a little random "jitter" to the delay when many clients retry at once.

CancelledError is special

asyncio.CancelledError inherits from BaseException, not Exception (since Python 3.8). So except Exception: does not swallow cancellation, which is deliberate. Never write a bare except: or except BaseException: without re-raising, because it breaks cancellation and timeouts. If you catch CancelledError for cleanup, re-raise it. The creating tasks lesson showed the pattern.

An exception inside a callback is only logged

Exceptions raised in plain callbacks (call_soon, add_done_callback) cannot be caught by your coroutines, because no coroutine is waiting for them. The loop logs them via its exception handler. You can install your own with loop.set_exception_handler(handler) to send such errors to your monitoring system.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A task raises an exception and nobody awaits it. What happens?

    Failures are stored in the task. If never observed, asyncio logs it later, which is easy to miss.
  2. 2. Which does NOT retrieve a task's exception?

    done() only reports whether the task has finished; it does not look at the exception.
  3. 3. Why does except Exception not catch CancelledError?

    Since Python 3.8 CancelledError derives from BaseException so generic handlers do not swallow cancellation.
  4. 4. What is a good retry strategy for a flaky network call?

    Bounded retries with exponential backoff avoid hammering a struggling service and still report persistent failures.

Summary

  • A failing task stores its exception; you see it when you await the task or call result() / exception().
  • Unobserved failures are only logged as "never retrieved", so always await tasks or attach a logging callback.
  • Catch errors around the await with ordinary try/except/finally.
  • Retry flaky calls with bounded exponential backoff; never swallow CancelledError.

Next: Locks and race conditions