Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 36 ยท Integration and Deployment

Celery with Flask and FastAPI: Task Status APIs

Integrate Celery with Flask (app factory, ContextTask) and FastAPI (202 Accepted, task status endpoints), and choose between BackgroundTasks and Celery.

Flask and FastAPI do not ship a background-job system. Celery fills that gap: the web process accepts a request, queues a job and answers at once, and a worker does the slow part. The pattern is the same in both frameworks: start the task, return its id, let the client ask for status later. Flask adds one detail (the application context), FastAPI one decision (when BackgroundTasks is enough).

Flask, FastAPI, Celery and a broker do not run on this site. Framework code is read-only. The runnable examples are plain-Python stand-ins for the status-API contract and the context-aware task base class.

Flask: app factory and application context

Tasks often need Flask extensions (SQLAlchemy sessions, config) that expect an application context. The recommended approach is a Celery app created from the Flask app, with a task base class that pushes the context:

Python
# app.py
from celery import Celery, Task
from flask import Flask

def celery_init_app(app: Flask) -> Celery:
    class FlaskTask(Task):
        def __call__(self, *args, **kwargs):
            with app.app_context():
                return self.run(*args, **kwargs)

    celery_app = Celery(app.name, task_cls=FlaskTask)
    celery_app.config_from_object(app.config["CELERY"])
    celery_app.set_default()
    app.extensions["celery"] = celery_app
    return celery_app

def create_app() -> Flask:
    app = Flask(__name__)
    app.config.from_mapping(
        CELERY=dict(
            broker_url="redis://localhost:6379/0",
            result_backend="redis://localhost:6379/1",
            task_ignore_result=False,
        ),
    )
    celery_init_app(app)
    return app

Tasks use @shared_task so they do not import the app:

Python
# tasks.py
from celery import shared_task

@shared_task(bind=True)
def build_export(self, user_id):
    user = db.session.get(User, user_id)          # works: app context is active
    return make_zip(user)

Start the worker with the factory: celery -A app:celery_app worker after exposing celery_app = create_app().extensions["celery"] in a module (the exact -A target depends on your layout). Views start and poll tasks:

Python
@app.post("/exports")
def start_export():
    result = build_export.delay(current_user.id)
    return {"task_id": result.id}, 202

@app.get("/exports/<task_id>")
def export_status(task_id):
    result = AsyncResult(task_id)
    return {"state": result.state, "ready": result.ready(),
            "value": result.result if result.successful() else None}
Python Stand-in: a task base class that wraps a context

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

FastAPI: 202 Accepted and a status endpoint

FastAPI is asynchronous, but Celery's delay and result reads are normal blocking calls to Redis. They are quick, so calling them from a regular def endpoint (run in FastAPI's thread pool) is fine:

Python
from celery.result import AsyncResult
from fastapi import FastAPI
from .worker import celery_app, process_document

api = FastAPI()

@api.post("/documents", status_code=202)
def submit(document_id: int):
    result = process_document.delay(document_id)
    return {"task_id": result.id, "status_url": f"/tasks/{result.id}"}

@api.get("/tasks/{task_id}")
def task_status(task_id: str):
    result = AsyncResult(task_id, app=celery_app)
    body = {"task_id": task_id, "state": result.state}
    if result.state == "SUCCESS":
        body["result"] = result.result
    elif result.state == "FAILURE":
        body["error"] = str(result.result)
    return body

The contract is deliberately boring and works for any client: 202 Accepted with an id; GET the status until SUCCESS or FAILURE; use polling with backoff or push the finish event over WebSocket/Server-Sent Events from your own signal handler. Never make the HTTP request wait for the result.

Two details about states: an unknown id looks like PENDING (Celery cannot tell "queued" from "never existed"), so do not treat PENDING as proof the task exists. Keep your own job table if you need precise tracking. And custom progress can be reported with self.update_state(state="PROGRESS", meta={"done": 40, "total": 100}), which the status endpoint returns while the task runs.

Python Stand-in: submit then poll a task status

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Flask and FastAPI: when is Celery too much?

FastAPI's built-in BackgroundTasks runs a function after the response in the same process. Compare:

Need BackgroundTasks Celery
Quick fire-and-forget (log an event, send one email) Good Overkill
Survives a crash or deploy No Yes (message stays in the broker)
Retries, schedules, rate limits No Yes
Runs on separate machines, scales independently No Yes
Long jobs (minutes) Blocks that web worker Yes

Rule: if losing the job on a restart is acceptable and it takes under a second or two, BackgroundTasks is simpler. Otherwise use Celery. Alternatives such as ARQ or Dramatiq fit async-first code, but Celery's ecosystem (canvas, beat, Flower) is the broadest.

Common problems

  1. "Working outside of application context" in Flask: the task base class above was not used.
  2. Task not found: the worker never imported the tasks module. Use include=["tasks"] or celery_app.autodiscover_tasks([...]).
  3. Blocking the event loop: in async def endpoints do not call result.get(); use plain def endpoints or run_in_threadpool.
  4. Different serializers or config between web and worker: load the same config module in both.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does a Flask integration define a Task subclass that enters app.app_context()?

    Flask extensions need an application context; the base class pushes one around each task run.
  2. 2. Which HTTP response fits starting a long job?

    202 means accepted for processing; the client polls the status URL.
  3. 3. What state does AsyncResult report for an id Celery has never seen?

    PENDING is the default for unknown ids, so it cannot prove the task was queued.
  4. 4. When is FastAPI BackgroundTasks enough instead of Celery?

    BackgroundTasks run in the web process after the response, with no persistence or retries.

Summary

  • Flask: build the Celery app from the Flask app and use a Task subclass that pushes app.app_context().
  • FastAPI: return 202 and a status URL, read state with AsyncResult, and avoid blocking the event loop.
  • Unknown task ids look PENDING; report progress with update_state.
  • Use BackgroundTasks only for quick, loss-tolerant work; use Celery for durable, retried, scheduled or heavy jobs.

Next: Deploying Celery with Docker and systemd