Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 40 ยท Tasks and Gathering

asyncio.wait and as_completed: Handle Results as They Arrive

Learn asyncio.wait with FIRST_COMPLETED and ALL_COMPLETED, asyncio.as_completed to process results as they finish, and how to race tasks for the fastest answer.

gather returns everything at the end, in the order you asked. Often that is not what you want. Maybe you want to show each result the moment it arrives, or take the first answer and ignore the rest, or give up on stragglers after a deadline. asyncio.as_completed and asyncio.wait are the tools for those jobs.

as_completed: results in completion order

asyncio.as_completed(awaitables) yields results in the order they finish, fastest first. Use it with async for (Python 3.13+) or with a normal for loop that awaits each item (works on all versions):

Python Processing results as they finish

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The output is small.txt, then medium.png, then big.zip, because that is the order of completion. The program can update a progress bar or start working with each file immediately, instead of waiting for the slowest one.

Because completion order differs from submission order, you often need to know which job produced a result. Return the identifier together with the value:

Python Keeping track of which job finished

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You can also pass a timeout= to as_completed; if the deadline passes before the next result, awaiting raises TimeoutError.

asyncio.wait: split into done and pending

asyncio.wait(tasks, return_when=...) waits and then returns two sets: the tasks that are done and the ones still pending. It does not return results; you read them from the done tasks. The return_when argument controls when it stops waiting:

Value wait returns when...
ALL_COMPLETED (default) every task is finished
FIRST_COMPLETED at least one task finished (or failed or was cancelled)
FIRST_EXCEPTION any task raised, or all are done

Since Python 3.11 you must pass tasks or futures, not bare coroutines. Wrap coroutines with create_task first.

Racing tasks: take the first result

FIRST_COMPLETED is the way to race several sources, such as asking three mirror servers and using the quickest answer. Cancel the losers afterwards, otherwise they keep running:

Python Racing with FIRST_COMPLETED

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Note the cancel-then-gather cleanup: cancelling alone only requests cancellation, and awaiting the tasks lets them actually finish unwinding. Skipping it can leave "Task was destroyed but it is pending" warnings.

Waiting with a time limit

wait accepts timeout=. When time runs out it simply returns, with the unfinished tasks in pending; it does not cancel them or raise. That makes it useful for "give everyone one second, then see who made it":

Python wait with a timeout

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For timeouts around a single awaitable, asyncio.timeout() is simpler; see the timeouts chapter.

A worker pool with FIRST_COMPLETED

A loop around wait(FIRST_COMPLETED) keeps a fixed number of jobs in flight, starting a new one whenever another finishes. This is a hand-written concurrency limit:

Python Keep at most 2 jobs running

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In real code a semaphore or a queue with workers is shorter and clearer, but this example shows how wait composes.

Choosing between the tools

You want... Use
All results, in argument order gather
All-or-nothing with cancellation TaskGroup
Each result as it arrives as_completed
The first result only wait(..., FIRST_COMPLETED)
To see who finished before a deadline wait(..., timeout=...)

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In what order does as_completed produce results?

    as_completed hands out the fastest result first.
  2. 2. What does asyncio.wait return?

    You read each result from the done tasks; pending tasks are still running.
  3. 3. After wait(..., FIRST_COMPLETED), what should you do with the pending tasks you do not need?

    They keep running otherwise. Cancel them and then await (for example with gather) so they can unwind.
  4. 4. wait(tasks, timeout=0.3) times out. What happens?

    wait never raises on timeout and never cancels; it just returns the two sets.

Summary

  • as_completed yields results in completion order; return an identifier with each value.
  • wait returns (done, pending) sets and accepts ALL_COMPLETED, FIRST_COMPLETED, FIRST_EXCEPTION and timeout.
  • Race sources with FIRST_COMPLETED, then cancel and await the losers.
  • Pass tasks (not bare coroutines) to wait.

Next: Exceptions in tasks