Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 37 ยท Queues and Exchanges

Queues in Depth

Master RabbitMQ queues: declare options, durable vs persistent, exclusive and auto-delete queues, server-named queues, FIFO behavior and consumer prefetch.

A queue is where messages wait. It is the only RabbitMQ object that actually stores messages, so most operational questions (Will it survive a restart? Who may use it? How big can it grow?) are answered by the settings you give a queue when you declare it.

Declaration calls with pika are read-only examples for a real broker. The runnable blocks are Python stand-ins that reproduce the rules so you can experiment with them. They are not RabbitMQ.

Declaring a queue

Python
# real pika code, read-only
channel.queue_declare(
    queue="invoices",
    durable=True,        # survive a broker restart
    exclusive=False,     # usable by other connections
    auto_delete=False,   # keep it after the last consumer leaves
    arguments={"x-queue-type": "quorum"},
)

Declaring does three things: it creates the queue if it does not exist; it does nothing if it already exists with the same properties; it fails with PRECONDITION_FAILED if it exists with different ones. That makes queue_declare safe to call at application start.

Queue names are up to 255 bytes of UTF-8. Names starting with amq. are reserved for the broker.

Durable is not the same as persistent

Two separate switches decide whether a message survives a restart, and both must be on:

  • Durable queue (durable=True): the queue definition itself is kept and re-created after a restart.
  • Persistent message (delivery_mode=2): the message is written to disk.
Queue Message After a broker restart
durable persistent queue and message survive
durable transient queue survives, message is gone
transient persistent queue is gone, so the message is gone
transient transient everything is gone

The stand-in applies exactly this table.

Python Durable queue + persistent message (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Even a persistent message in a durable queue is not instantly safe: the broker writes it shortly after receiving it. To know when it is truly safe you need publisher confirms, covered in the reliability chapter.

Exclusive queues

An exclusive queue can be used only by the connection that declared it, and it is deleted when that connection closes. Exclusive queues are perfect for private reply queues and temporary subscriptions. They cannot be durable-and-shared; in RabbitMQ 4, exclusive queues are always classic queues and are transient.

Auto-delete queues

An auto-delete queue is removed once its last consumer unsubscribes. It never deletes itself if it never had a consumer. Use it for temporary work that should clean up after itself.

Server-named queues

If you pass an empty name, the broker invents one, such as amq.gen-JzTY20BRgKO-HjmUJj0wLg:

Python
# real pika code, read-only
result = channel.queue_declare(queue="", exclusive=True)
my_queue = result.method.queue      # the generated name

This pattern (empty name plus exclusive=True) gives each subscriber a private, throw-away queue. You will use it with fanout exchanges.

Rules in code

The stand-in below implements queue_declare with the three outcomes (create, no-op, precondition failure), plus exclusive ownership.

Python queue_declare rules (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Passive declaration

A passive declare (passive=True) only checks that the queue exists. It never creates anything and raises NOT_FOUND (closing the channel) if the queue is missing. It is handy for checking a queue's message and consumer counts:

Python
# real pika code, read-only
info = channel.queue_declare(queue="invoices", passive=True)
print(info.method.message_count, info.method.consumer_count)

Ordering and delivery

A queue is a FIFO: messages are delivered in the order they arrived. Several things loosen strict order: multiple consumers working in parallel, a message that is rejected and requeued, and priorities. Every delivery has a delivery_tag, and consumers should acknowledge it when the work is finished.

Prefetch (QoS)

If a queue holds a thousand messages and a consumer attaches, the broker would happily push all of them. Prefetch limits how many unacknowledged messages a consumer may hold at once:

Python
# real pika code, read-only
channel.basic_qos(prefetch_count=10)

A small prefetch spreads work fairly among workers; a larger one keeps a fast consumer busy and improves throughput. The stand-in shows the fairness difference with one slow and one fast worker.

Python Prefetch and fair dispatch (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With a large prefetch the slow worker grabs a big share up front and the total time suffers; with prefetch 1 jobs go to whichever worker is free. (This is a simplified model of the real behavior.)

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What do you need for a message to survive a broker restart?

    The queue must be durable to be re-created and the message persistent to be written to disk. Both are required.
  2. 2. When is an auto-delete queue removed?

    Auto-delete queues are deleted once they have had at least one consumer and the last one is gone.
  3. 3. What does queue_declare do for an existing queue with different arguments?

    Declaring is idempotent only for identical settings. A mismatch is an error that closes the channel.
  4. 4. How do you get a private, throw-away queue with a generated name?

    An empty name asks the broker to generate one, and exclusive=True makes the queue private to your connection.
  5. 5. What does basic_qos prefetch_count limit?

    Prefetch bounds in-flight unacknowledged deliveries per consumer or channel, which controls fairness and memory use.

Summary

  • queue_declare creates, no-ops or fails with PRECONDITION_FAILED depending on matching properties.
  • Survival across restarts needs a durable queue and a persistent message.
  • Exclusive queues are private to one connection; auto-delete queues vanish after their last consumer; an empty name gives a server-generated one.
  • Passive declare checks existence and counts.
  • Prefetch limits unacknowledged messages and shapes fairness and throughput.

Next: Exchanges Overview