Skip to content
Tutorials Center
My learning

Topics

Lesson 7 of 36 ยท Tasks and Workers

Task Results, States and AsyncResult

Understand Celery task states (PENDING, STARTED, SUCCESS, FAILURE, RETRY), how to read results with AsyncResult, when to ignore results and how to avoid blocking.

Every task call gives you an AsyncResult: a small handle that remembers the task id. It does not hold the value itself. When you ask it a question (state, ready(), get()), it goes to the result backend and looks the answer up. That is why a result backend is optional: without one you can still run tasks, you just cannot ask about them.

The task life cycle

stateDiagram-v2
    [*] --> PENDING
    PENDING --> STARTED: worker begins (if track_started)
    PENDING --> SUCCESS
    STARTED --> SUCCESS
    STARTED --> FAILURE
    STARTED --> RETRY
    RETRY --> STARTED
    PENDING --> REVOKED
    SUCCESS --> [*]
    FAILURE --> [*]

Built-in states. RECEIVED and REJECTED also exist but are mostly seen in events.

State Meaning
PENDING Unknown to the backend: queued, or the id does not exist.
STARTED A worker began running it. Only recorded when task_track_started=True.
SUCCESS Finished; result holds the return value.
FAILURE Raised an exception; result holds the exception.
RETRY Failed and is scheduled to run again.
REVOKED Cancelled with revoke() or expired.

States that end a task (SUCCESS, FAILURE, REVOKED) are called ready or final.

Reading a result

Python
from celery.result import AsyncResult

res = add.delay(2, 3)
res.id                  # '6f2c1f0e-...'
res.state               # 'PENDING'
res.ready()             # False until final
res.successful()        # True/False once ready
res.get(timeout=10)     # 5, waits at most 10 s

Because the handle is just an id, you can rebuild it anywhere, even in a different process:

Python
res = AsyncResult(task_id, app=app)
if res.ready():
    ...

This is the standard pattern for web apps: the view starts a task and returns {"task_id": ...}; the browser polls a status endpoint that builds an AsyncResult(task_id) and reports state.

What get() does with errors

If the task raised an exception, get() re-raises it in the caller (propagate=True by default). Pass propagate=False to receive the exception object as the return value instead, and inspect res.traceback for the worker-side traceback as text. If the task takes longer than timeout, get() raises celery.exceptions.TimeoutError. That is not a task failure; the task keeps running.

The stand-in below models a result backend and AsyncResult with the same behaviour for get(propagate=...), ready(), and timeout.

Python Stand-in: AsyncResult against a tiny backend

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that the unknown id reports PENDING, and that a timeout (task c is still STARTED) is a different exception from a task failure (task b).

Tracking progress with custom states

Besides the built-in states you may publish your own from inside a task using update_state. It is the standard way to show a progress bar:

Python
@app.task(bind=True)
def import_rows(self, path):
    rows = load(path)
    for i, row in enumerate(rows, start=1):
        save(row)
        self.update_state(state="PROGRESS", meta={"done": i, "total": len(rows)})
    return {"done": len(rows)}

A status endpoint then reads res.state and, for PROGRESS, res.info (the meta dict). Updating on every single row hammers the backend; update every N rows or every second.

When you do not need results

Storing every return value costs memory and network round trips. Many tasks (send an email, clear a cache) return nothing useful. Turn results off:

Python
@app.task(ignore_result=True)
def send_email(user_id): ...

# or for the whole app
app.conf.task_ignore_result = True

With results ignored, AsyncResult(id).state will stay PENDING forever, so do not combine this with status polling. Also remember:

  • Results expire (result_expires, one day by default). An old id returns PENDING.
  • Call res.forget() to delete a result you have consumed, or res.get() followed by forget() in tight loops.
  • Never return large objects. Store files or big data elsewhere (a database, S3) and return the key.
  • Failures are stored too, including the exception type and traceback, which is great for debugging but means exceptions must be picklable or serializable by the result serializer.

Do not poll res.get() or res.ready() in a tight loop in a web request. Return the task id, let the client poll a lightweight status endpoint, or push the result to the browser with WebSockets.

Do not wait inside tasks

Calling other.delay().get() from inside a task blocks one worker slot waiting for another. If every slot ends up waiting, nothing can run and the system deadlocks. Celery warns about this unless you explicitly pass disable_sync_subtasks=False. Use chains or callbacks instead.

A polling endpoint, step by step

Here is the whole pattern you will use in web apps, as a read-only sketch:

Python
@app.post("/reports")
def start_report():
    res = build_report.delay(current_user.id)
    return {"task_id": res.id}, 202

@app.get("/reports/<task_id>")
def report_status(task_id):
    res = AsyncResult(task_id, app=celery_app)
    if res.state == "PROGRESS":
        return {"state": res.state, **res.info}
    if res.successful():
        return {"state": res.state, "url": res.result["url"]}
    if res.failed():
        return {"state": res.state, "error": "report failed"}, 500
    return {"state": res.state}

Notice the 202 Accepted response: it is the HTTP way to say "I took your request but it is not finished".

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does an AsyncResult object actually contain?

    It is a handle. State and value are fetched from the result backend when you ask.
  2. 2. A task that raised ValueError is read with res.get(). What happens by default?

    propagate=True re-raises the stored exception. Use propagate=False to receive it as a value.
  3. 3. You set ignore_result=True and poll the state of the task. What do you see?

    With results ignored there is nothing in the backend, so the state stays at the default PENDING.
  4. 4. Why is other_task.delay().get() inside a task discouraged?

    If all worker slots are waiting for results that need a free slot, nothing can progress. Use chains or callbacks.

Summary

  • AsyncResult is a handle keyed by task id; state, ready(), get() query the backend.
  • States: PENDING (unknown), STARTED, SUCCESS, FAILURE, RETRY, REVOKED; custom states like PROGRESS use update_state.
  • get() re-raises task errors unless propagate=False, and raises a timeout error if the task is just slow.
  • Use ignore_result=True when you do not need values, expect results to expire, and never block on results in a web request or inside a task.

Next: Running Workers: Pools, Concurrency and Prefetch