Skip to content
Tutorials Center
My learning

Topics

Lesson 3 of 36 ยท Getting Started

Installing Celery and Running Your First Task

Install Celery, start Redis, write your first task, run a worker and call the task with delay(). A step-by-step guide with a runnable stand-in of the same flow.

In this lesson you will go from an empty folder to a task that a worker executes. The real steps need a Redis server and a terminal, so the commands are read-only here. A runnable stand-in at the end reproduces the whole flow in your browser so you can see what each step does.

Step 1: install Celery

Create a virtual environment and install Celery. Python 3.9 or newer is required by current releases; check the release notes for your version.

Terminal
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install "celery[redis]"

The part in brackets is an extra. celery[redis] installs Celery together with the Redis client library. Other useful extras are celery[amqp]-style transports for RabbitMQ (installed by default through kombu/amqp), celery[sqs] for Amazon SQS and celery[msgpack] for a compact serializer.

Step 2: start a broker

The easiest local broker is Redis. With Docker:

Terminal
docker run -d --name redis -p 6379:6379 redis:7

Or install Redis with your package manager and run redis-server. Test it:

Terminal
redis-cli ping      # prints PONG

Step 3: write the app and a task

Create tasks.py:

Python
from celery import Celery

app = Celery(
    "tasks",
    broker="redis://localhost:6379/0",
    backend="redis://localhost:6379/1",
)

@app.task
def add(x, y):
    return x + y

What each piece means:

  • Celery("tasks", ...): the first argument is the main name. When a task is defined in a module run as __main__, Celery uses this name to build task names, so keep it equal to the module name.
  • broker=...: where messages go. Database 0 of the local Redis.
  • backend=...: where results go. Database 1, so results and messages do not share a keyspace.
  • @app.task: registers the function as a task named tasks.add.

Step 4: start a worker

In a second terminal, in the same folder:

Terminal
celery -A tasks worker --loglevel=INFO

-A tasks means "the application is in the module tasks". The worker prints a banner listing the app, the broker, the concurrency (one process per CPU core by default) and the registered tasks:

TEXT
[tasks]
  . tasks.add

If your task is missing from that list, the worker did not import your module. That is the most common beginner problem.

Step 5: call the task

Open a Python shell in the same folder:

Python
>>> from tasks import add
>>> result = add.delay(4, 4)
>>> result
<AsyncResult: 6f2c...>
>>> result.get(timeout=10)
8

In the worker terminal you will see a line stating that tasks.add[<id>] was received and later succeeded with the result 8. Calling add(4, 4) without .delay would just run the function in your shell, as normal Python.

result.get() blocks until the task is finished. Use it in shells and tests. In a web request, blocking on get() defeats the purpose of the queue; return the task id instead and let the client check later.

The same flow, in your browser

This stand-in implements a miniature Celery class with task, delay, a worker loop and a results dict. Run it, then change delay(4, 4) to different numbers.

Python Stand-in: a miniature Celery app

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The output makes the difference between add(1, 2) and add.delay(4, 4) obvious: the first returns 3 straight away, the second only returns a handle, and the value appears after the worker loop has run.

What goes wrong most often

Symptom Likely cause
Connection refused when calling .delay() The broker is not running, or the URL is wrong.
Worker starts but the task is not in the [tasks] list The module was not imported: wrong -A value or the file is not on the path.
Message stays in the queue forever No worker is running, or the worker listens to a different queue.
Received unregistered task of type ... The worker's code does not define that task name, often because it was started before you added the task. Restart it.
result.get() hangs No result backend is configured or no worker is running.
Changing the code has no effect You did not restart the worker.

Arguments must be serializable

Because delay() turns the call into JSON, passing a non-JSON value fails at the producer, before anything is queued. This runnable check shows the same effect with the standard json module:

Python Why delay() accepts ids but not objects

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In real Celery the error is kombu.exceptions.EncodeError, with a similar message. The fix is the same: pass an ISO string, a number or an id, and rebuild the object in the worker.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the -A tasks option of the celery command mean?

    -A (--app) tells the command where to find the Celery app instance, here the module tasks.
  2. 2. You call add.delay(4, 4) but nothing happens. The worker is running and shows no task list entry for add. What is the likely problem?

    A worker only knows tasks it has imported. Check the [tasks] list in its banner and the -A argument.
  3. 3. Why use different Redis databases (/0 and /1) for broker and backend in the example?

    It is only a convenience. Separate databases make it easier to inspect or flush one without touching the other.
  4. 4. What does add(1, 2) (without delay) do?

    A task object is still callable. Calling it directly bypasses the broker and workers entirely.

Summary

  • Install celery[redis], start Redis, define an app with a broker and a backend.
  • Decorate a function with @app.task, start a worker with celery -A module worker, call .delay().
  • delay() returns an AsyncResult; get() waits for the value.
  • Most first-time errors are a missing broker, an unimported module or a worker that was not restarted.
  • Arguments must be JSON-friendly, so pass ids and strings.

Next: Project Layout, App Configuration and Autodiscovery