Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 37 ยท Reliability and Acknowledgements

Consumer Acknowledgements and Prefetch

Understand RabbitMQ manual acknowledgements, basic.nack, requeueing, redelivery and the prefetch (QoS) limit, with pika examples and a runnable delivery simulation.

A broker that hands a message to a consumer faces a question: when may it forget the message? If it deletes it the moment it is sent, a consumer crash loses the work. RabbitMQ solves this with consumer acknowledgements. A delivered message stays on the queue, marked unacknowledged, until the consumer says it is done. If the consumer's channel or connection closes first, the broker puts the message back and delivers it again, possibly to another consumer.

This lesson covers the three acknowledgement methods, the prefetch limit that controls how many unacked messages a consumer may hold, and the timeouts and mistakes that cause stuck queues. Publisher-side guarantees are in Publisher Confirms and Returns.

RabbitMQ is not installed on this page, so the pika examples are read-only. The runnable blocks are a plain Python stand-in for the delivery state machine: one queue, a "ready" list and an "unacked" table. It illustrates the rules but is not RabbitMQ.

The delivery life cycle

stateDiagram-v2
    [*] --> Ready: published
    Ready --> Unacked: basic.deliver
    Unacked --> [*]: basic.ack
    Unacked --> Ready: nack or reject requeue=true
    Unacked --> Dead: nack or reject requeue=false
    Unacked --> Ready: channel closed
    Dead --> [*]: dropped or dead-lettered

A message has a delivery tag, a per-channel counter that identifies each delivery. You acknowledge by tag, on the same channel that received the message. Acknowledging on another channel is a protocol error.

Automatic versus manual acknowledgement

With auto_ack=True (called no_ack in the protocol), the broker considers a message handled as soon as it writes it to the socket. This is the fastest mode and also the least safe: a crash mid-processing loses the message, and a slow consumer can be flooded because nothing limits how much is in flight. Use it only for data you can lose.

With manual acknowledgements you call one of:

Method Effect
basic_ack(tag, multiple=False) Success. The message is removed. multiple=True acks every earlier unacked tag on the channel too.
basic_nack(tag, multiple=False, requeue=True) Failure. Can cover many tags at once. RabbitMQ extension to AMQP 0-9-1.
basic_reject(tag, requeue=True) Failure for one message. Standard AMQP.

With requeue=True the message returns to the queue and is usually redelivered at its original position (classic queues try to preserve order). With requeue=False it is dropped, or sent to a dead letter exchange if the queue has one (next lessons: Dead Letter Exchanges and Retries).

Python
import pika

connection = pika.BlockingConnection(pika.ConnectionParameters("localhost"))
channel = connection.channel()
channel.queue_declare(queue="tasks", durable=True)
channel.basic_qos(prefetch_count=10)

def handle(ch, method, properties, body):
    try:
        process(body)
        ch.basic_ack(delivery_tag=method.delivery_tag)
    except TemporaryError:
        ch.basic_nack(method.delivery_tag, requeue=True)
    except PermanentError:
        ch.basic_nack(method.delivery_tag, requeue=False)  # to the DLX

channel.basic_consume(queue="tasks", on_message_callback=handle)
channel.start_consuming()

The classic bug: forgetting to call basic_ack. The consumer works, the queue seems fine, but "Unacked" grows forever and memory climbs. When the consumer finally disconnects, every message is redelivered. Always ack, even in error paths.

The redelivered flag

When a message is delivered again after a requeue or a lost channel, the broker sets redelivered=true on the delivery. It means "this message may have been seen before", not "this message failed". A consumer can use it to be more careful: for example, check a database of processed ids before doing the work again. Quorum queues additionally keep an x-delivery-count header that grows with every return.

Python Stand-in: ack, nack, requeue and crash

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You should see b come back at the head of the queue with the redelivered flag set, c end up in the dead list, and the unacked message returned when the channel closes.

Prefetch: how much can be in flight

By default a consumer has unlimited prefetch: the broker pushes every ready message to it as fast as the network allows. That is rarely what you want. basic.qos sets a prefetch count, the maximum number of unacknowledged messages per consumer (or per channel, with global=True). When the limit is reached the broker stops sending until acks arrive.

Choosing a value:

  • prefetch = 1: perfect fairness for long, uneven tasks, but each message costs a round trip, so throughput for fast tasks is poor.
  • small tens (10 to 50): a good starting point for most services.
  • hundreds or more: only for very fast handlers; huge values pull messages away from other consumers and hide backlog.

A rough formula: prefetch โ‰ˆ round-trip time / processing time per message, with a floor that keeps the consumer busy. The Work Queues and Fair Dispatch lesson shows its effect on balance.

Python Stand-in: prefetch limit in action

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With a high prefetch the slow consumer grabs half the work up front and everyone waits for it. With prefetch 1 the fast consumer does most of the jobs and the total time drops sharply.

Timeouts and other traps

  • Delivery acknowledgement timeout. Since RabbitMQ 3.8.15 a consumer that holds a delivery unacked for longer than consumer_timeout (default 30 minutes) gets its channel closed with PRECONDITION_FAILED. Long jobs should ack early (and track progress elsewhere) or raise the timeout.
  • Double ack. Acking the same tag twice is a channel error (unknown delivery tag).
  • Requeue loops. nack(requeue=True) on a message that always fails creates a hot loop. Use a dead letter exchange, a delivery limit (quorum queues) or a retry delay instead.
  • Heartbeats and blocking handlers. pika.BlockingConnection only processes heartbeats while it runs its I/O loop, so a long synchronous handler can get the connection dropped. Use a worker thread or connection.process_data_events().

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A consumer using manual acks crashes while holding three unacked messages. What happens to them?

    When the channel or connection closes, the broker requeues every unacked message and marks later deliveries as redelivered.
  2. 2. What does basic_nack(tag, requeue=False) do on a queue with no dead letter exchange?

    With requeue=false the message is removed. If the queue has a dead letter exchange configured, it is republished there instead.
  3. 3. Which prefetch setting gives the fairest distribution for long and uneven tasks?

    A prefetch of 1 gives a new message only when the previous one is acknowledged, so slow consumers do not hoard work.
  4. 4. What is the most common symptom of a consumer that never calls basic_ack?

    Delivered but unacknowledged messages stay on the broker, counted as Unacked, until the channel closes.

Summary

  • Manual acknowledgements protect against consumer crashes; messages stay unacked until ack, nack or reject.
  • requeue=True retries, requeue=False discards or dead-letters; the redelivered flag warns of a possible duplicate.
  • basic.qos prefetch bounds in-flight messages; small values balance load, larger ones raise throughput.
  • Watch the 30-minute delivery timeout, forgotten acks and requeue loops.

Next: Durability and Persistent Messages