Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 57 ยท Messaging and Integration

Service Bus Queues and Topics

Learn Azure Service Bus: queues, topics and subscriptions, peek-lock, dead-letter queues, sessions and duplicate detection, with runnable stand-ins.

When one part of a system needs another to do work, the simplest design is a direct call: the order service calls the billing service and waits. That design is fragile. If billing is slow, orders are slow; if billing is down, orders fail; if traffic spikes, billing is overwhelmed. A message broker fixes this by placing a durable buffer between the two. The sender puts a message in and moves on; the receiver takes messages out at its own pace, and nothing is lost if it is briefly offline.

Azure Service Bus is Azure's enterprise message broker. It offers queues (one message goes to one consumer) and topics with subscriptions (one message goes to many consumers), and guarantees that matter in business systems: reliable delivery, ordering when you ask for it, dead-lettering of poisonous messages and duplicate detection. The real service needs an Azure namespace, so the SDK code below is read-only. The runnable stand-ins are Node.js programs that model the key behaviors with a fake clock: peek-lock, lock timeouts and dead-lettering, topic filters and duplicate detection.

Queues

A queue holds messages until a consumer processes them. Several consumers can read the same queue (the competing consumers pattern), but each message is delivered to just one of them. A queue gives you:

  • Load leveling: a burst of 10,000 orders queues up, and workers drain it at their own speed.
  • Decoupling: the sender does not know who or how many receivers exist, or whether they are running.
  • Reliability: messages are stored durably until they are completed, expire or are dead-lettered.

You work within a namespace (a container with its own DNS name such as mycompany.servicebus.windows.net) in one of three tiers: Basic (queues only), Standard (queues, topics and subscriptions) and Premium (dedicated capacity, larger messages, more isolation). Standard messages are limited to 256 KB, so send IDs or links to large payloads instead of the payloads themselves; check the current limits for Premium.

Creating a queue with the CLI (read-only; parameter names can change between versions):

Terminal
az servicebus namespace create --resource-group integ-rg --name mycompany-bus \
  --location westeurope --sku Standard

az servicebus queue create --resource-group integ-rg --namespace-name mycompany-bus \
  --name orders --max-delivery-count 5 --lock-duration PT1M

Peek-lock: how reliable delivery works

A receiver can read in one of two modes:

  • Receive-and-delete: the broker removes the message the moment it hands it over. Fast, but if the receiver crashes before finishing, the message is lost ("at most once").
  • Peek-lock (the usual choice): the broker hands the message over and locks it, hiding it from other receivers for the lock duration (30 seconds by default, up to 5 minutes). The receiver then settles the message:
    • Complete: processed; the broker deletes it.
    • Abandon: give it back immediately; its delivery count goes up.
    • Dead-letter: move it to the dead-letter queue.
    • Defer: set it aside to be fetched later by its sequence number.

If the receiver crashes or takes longer than the lock duration, the lock expires and the message becomes visible again, to be delivered to someone else. This gives at least once delivery, which means that your handler must be idempotent: processing the same message twice must be harmless.

stateDiagram-v2
    [*] --> Active: "Sender enqueues"
    Active --> Locked: "Receiver takes it (peek-lock)"
    Locked --> Completed: "Complete"
    Locked --> Active: "Abandon or lock expires<br/>delivery count + 1"
    Locked --> DeadLetter: "Dead-letter, or count exceeds max"
    Completed --> [*]

A message is either waiting, locked by one receiver, or finished. Failures send it back to waiting until it has been tried too many times.

The dead-letter queue

Every queue and subscription has a built-in dead-letter queue (DLQ), a sub-queue named $deadletterqueue. A message ends up there when:

  • its delivery count exceeds the max delivery count (default 10), usually because the handler keeps failing on it, a poison message;
  • its time to live expires, if you enabled dead-lettering on expiration;
  • the receiver dead-letters it explicitly with a reason and description;
  • a subscription filter evaluation fails.

The DLQ is not processed automatically. Someone must monitor it (alert on its message count), inspect the reason and either fix the cause and resubmit the message, or discard it.

Sending and receiving in code

Read-only Node.js with the @azure/service-bus and @azure/identity packages, authenticating with Microsoft Entra (roles such as Azure Service Bus Data Sender and Data Receiver) instead of a connection string:

JavaScript
import { ServiceBusClient } from "@azure/service-bus";
import { DefaultAzureCredential } from "@azure/identity";

const client = new ServiceBusClient(
  "mycompany-bus.servicebus.windows.net",
  new DefaultAzureCredential()
);

// Send
const sender = client.createSender("orders");
await sender.sendMessages({
  messageId: "order-1001",          // used by duplicate detection
  body: { orderId: 1001, total: 49.5 },
  contentType: "application/json",
});

// Receive with peek-lock
const receiver = client.createReceiver("orders", { receiveMode: "peekLock" });
const [message] = await receiver.receiveMessages(1, { maxWaitTimeInMs: 5000 });
if (message) {
  try {
    await processOrder(message.body);
    await receiver.completeMessage(message);
  } catch (err) {
    if (message.deliveryCount >= 3) {
      await receiver.deadLetterMessage(message, {
        deadLetterReason: "ProcessingFailed",
        deadLetterErrorDescription: String(err),
      });
    } else {
      await receiver.abandonMessage(message);
    }
  }
}

In practice you usually call receiver.subscribe(...) with a message handler and an error handler, and the SDK renews the lock for long-running handlers when you ask it to.

A queue with locks and dead-lettering: a stand-in

This stand-in models one queue with a 30-second lock, a maximum delivery count of 3 and a fake clock. A worker takes a message, and one invented message ("bad") always fails. A second message ("slow") takes longer than the lock, so it becomes visible again while the first worker is still busy, the classic cause of duplicate processing. It is a model of the rules, not of Service Bus.

Node.js Peek-lock, lock expiry and the DLQ

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

In the output, notice the bad message reaching its third delivery and moving to the DLQ without blocking the others, and the slow message being delivered a second time. The fixes for the latter are a longer lock, automatic lock renewal, or splitting the work.

Topics and subscriptions

A topic is like a queue with a fan-out. Senders publish to the topic; each subscription receives its own copy of every message that matches the subscription's rule, and subscribers read from their subscription as if it were a queue (with its own locks, dead-letter queue and delivery counts). Adding a new consumer means adding a subscription; the sender never changes.

Rules are filters:

  • SQL filter: a SQL-like expression over message properties, such as region = 'EU' AND amount > 100.
  • Correlation filter: an equality match on specific properties, which is faster.
  • Boolean filter: match everything or nothing. A new subscription gets a default rule that matches all.

Rules can also run an action, such as setting a property, and filters use message properties (the system and application properties), not the body.

Sessions and ordering

A queue does not guarantee strict order once there are competing consumers or retries. If you need related messages processed in order and by one consumer at a time, enable sessions on the queue or subscription and set a sessionId on each message, for example the customer ID. A receiver locks a whole session and processes its messages first in, first out; different sessions run in parallel. Use it when updates to one entity must not be reordered.

Duplicate detection

Senders retry, and a retried send can create two copies. If you enable duplicate detection when creating the queue or topic, the broker remembers the messageId of recent messages for a configurable window (10 minutes by default) and silently drops a second message with the same ID. It must be turned on at creation time, and you must choose stable IDs yourself, such as the business key order-1001, not a random value.

Node.js Topic filters and duplicate detection

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The last publish of order-2 at t=700 is outside the 600-second window, so it is accepted again. Duplicate detection narrows the problem but does not remove the need for idempotent handlers.

Common mistakes

  • Not handling poison messages, so a bad message blocks a worker in a retry loop. Set a max delivery count and watch the DLQ.
  • Handlers that are not idempotent under at-least-once delivery.
  • A lock shorter than the handler's run time, producing duplicates. Renew locks or lengthen them.
  • Using connection strings with full-access keys everywhere. Prefer Entra roles with the narrowest rights (sender versus receiver).
  • Choosing Service Bus when Event Hubs is the right tool. For firehose telemetry see Event Grid and Event Hubs; for simple, cheap queues see Azure Files, Queues and Tables.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In peek-lock mode, what happens if the receiver crashes before settling the message?

    The lock times out, the delivery count increases and another delivery is attempted. This is at-least-once delivery.
  2. 2. Which of these moves a message to the dead-letter queue?

    Exceeding max delivery count (a poison message), expiry with dead-lettering enabled, or an explicit dead-letter call all lead to the DLQ.
  3. 3. How does a topic differ from a queue?

    Subscriptions receive their own copy of matching messages. Topics need the Standard or Premium tier.
  4. 4. You need all updates for one customer processed in order, one at a time. Which feature do you use?

    A session groups related messages and a receiver locks the whole session, giving first-in, first-out processing per entity.
  5. 5. Duplicate detection relies on which message property?

    The broker compares messageId values inside the configured window. Stable business keys work best.

Summary

  • Service Bus is a durable message broker: queues for one consumer per message, topics with subscriptions for fan-out.
  • Peek-lock gives at-least-once delivery: complete, abandon, defer or dead-letter, and an expired lock redelivers the message. Make handlers idempotent.
  • Messages that exceed the max delivery count go to the dead-letter queue, which you must monitor.
  • Subscription rules (SQL, correlation, boolean) route messages by their properties.
  • Sessions give ordered, per-entity processing; duplicate detection drops repeated message IDs inside a window.
  • Prefer Entra authentication with separate sender and receiver roles.

Next: Event Grid and Event Hubs.