Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 36 ยท Projects and Practice

Celery Best Practices, Pitfalls and Practice Problems

A production checklist for Celery: small idempotent tasks, JSON payloads, timeouts, retries, queues, monitoring, plus practice exercises and interview-style questions.

You have now seen the whole shape of a Celery system: tasks, brokers, workers, scheduling, workflows, reliability, scaling and deployment. This closing lesson condenses the advice into a checklist, lists the mistakes that cause most production incidents, and gives you practice problems with solutions to test your understanding.

Celery does not run on this site. Celery snippets are read-only; the runnable exercises use plain-Python stand-ins, so you can check your reasoning about argument passing, backoff and idempotency.

Task design

  1. Small, focused tasks. One task, one job. Small tasks are easier to retry, time-limit, route and test, and they compose in chains and groups.
  2. Pass identifiers, not objects. Send ids, keys and paths; load fresh data in the task (Django integration).
  3. JSON-serializable arguments and results. Use the JSON serializer; never accept pickle from untrusted producers.
  4. Keep messages and results small. Large payloads slow the broker and the backend. Store data elsewhere and pass a reference.
  5. Idempotent by design. Assume every task can run twice (Acks late and idempotency).
  6. Never block on other tasks. Do not call .get() inside a task; compose with chains, chords or callbacks.
  7. Give tasks explicit names (@app.task(name="orders.send_receipt")). Automatic names change when modules move, and in-flight messages break.
  8. Ignore results you do not read (ignore_result=True) to save backend load.

Reliability

  1. Timeouts everywhere. soft_time_limit and time_limit on tasks, and timeout= on every network call (Time limits).
  2. Retry only transient errors, with exponential backoff and jitter (Retries).
  3. acks_late=True and reject_on_worker_lost=True for work you cannot lose, with a visibility timeout above your longest task on Redis.
  4. Handle final failure: error callbacks, dead letters and alerts (Failure handling).
  5. Publish after commit. Use transaction.on_commit or its equivalent.
  6. Set expires on time-sensitive tasks so a backlog does not trigger stale actions.

Operations

  1. Separate queues by latency class and resource type, each with dedicated workers (Queues and routing).
  2. Tune prefetch and concurrency to the workload, and recycle children (--max-tasks-per-child) (Concurrency).
  3. Monitor queue age, failure rate and worker count; secure Flower (Inspecting workers).
  4. Exactly one beat, with durable schedule storage (Deployment).
  5. Graceful shutdown with a stop timeout longer than your longest task.
  6. Version task signatures carefully: deploy code that accepts old and new arguments first, then change producers.
  7. Never flush the broker in a deploy script.

Common mistakes and their symptoms

Symptom Likely cause
Task never runs No worker consumes that queue; typo in the queue name; worker did not import the task module
DoesNotExist in a task Published before the transaction committed
Task runs twice acks_late or visibility timeout; retry after partial success; not idempotent
Worker memory grows forever Leaks, large results, no --max-tasks-per-child
Short tasks wait behind a long one Prefetch hoarding; shared queue for different workloads
Chord never fires No result backend, ignore_result=True on header tasks, or a failed header member
Periodic tasks run twice Two beat processes
Memory error on result read Returning huge objects from tasks
Tasks lost on deploy Early ack plus SIGKILL, or a stop timeout that is too short

Practice problems

Try each before opening the solution idea. The TryIts let you test your answers.

1. Argument merging

add.s(2) is used as the middle step of chain(add.s(1, 1), add.s(2), add.s(3)). What does the chain return, and what changes if the last step is add.si(5, 4)?

Python Check your answer: argument prepending

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

2. Choose a retry policy

A task calls a partner API that sometimes returns 503 and always returns 400 for bad input. Which exceptions go into autoretry_for? Why add jitter? Compute the worst-case waiting time for retry_backoff=2, retry_backoff_max=60, max_retries=6 without jitter.

Python Check your answer: backoff total

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Retry the 503 (a requests.HTTPError you raise only for 5xx, or a custom TemporaryError) and never the 400, because the second will fail identically. Jitter spreads simultaneous retries.

3. Make it idempotent

This task credits loyalty points. Redelivery adds points twice. Rewrite it to be safe.

Python Check your answer: idempotent credit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

4. Design questions (no code)

  1. A queue contains 1 million tiny tasks and CPU usage is low while the queue barely moves. What do you check first? (Concurrency, prefetch, message overhead: batch with chunks, or raise concurrency for I/O-bound work.)
  2. Users complain password-reset emails arrive an hour late after a nightly import. What is the structural fix? (Separate queue and workers for emails; priorities alone do not isolate.)
  3. After deploy, tasks fail with TypeError: unexpected keyword argument. What happened? (Old workers or old queued messages with a different signature: roll out in two compatible steps.)
  4. A chord body never runs. List three causes. (No backend, results ignored, a failed header member.)
  5. How do you cancel a 20-minute job cleanly? (Cancel flag checked between steps; soft time limit; revoke as a last resort.)

5. Interview-style quick answers

  • At-most-once vs at-least-once? Early ack vs acks_late; Celery never offers exactly-once.
  • Why not pass ORM objects? Not serializable, stale snapshots, large messages.
  • What does a result backend do? Stores states and return values, needed for get(), groups and chords.
  • Difference between a chain and a chord? Sequential pipe vs parallel group plus one callback with all results.
  • Why a separate queue for slow tasks? Isolation and independent scaling.
Python Stand-in: a tiny self-check for your task design

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Where to go next

  • Rebuild one of the projects (thumbnail pipeline, report fan-out) with a real broker on your machine and break it on purpose: kill workers, stop Redis, send duplicates.
  • Read your own worker logs during that experiment; they teach more than any diagram.
  • Compare Celery with lighter queues (RQ, Dramatiq, ARQ) and with workflow engines (Airflow, Temporal) to learn which problems each fits.

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. A process_order task occasionally credits a customer twice. What is the most robust fix?

    At-least-once delivery means duplicates happen; only idempotency makes them harmless.
  2. 2. Which is NOT a recommended practice?

    Blocking on another task can deadlock the worker pool; use chains, chords or callbacks.
  3. 3. Which problem is most likely if two beat processes run?

    Each beat publishes every due task, so you get duplicates. Run exactly one.

Summary

  • Design small, idempotent tasks that pass ids and small JSON payloads, with timeouts and selective retries.
  • Operate with separate queues, tuned workers, monitoring on queue age and failures, one beat and graceful shutdowns.
  • Most incidents trace back to the mistakes in the table: missing workers, publish-before-commit, non-idempotent tasks, and shared queues.
  • Practice by rebuilding the projects and breaking them deliberately.

Next: Back to the Celery course