Skip to content
Tutorials Center
My learning

Topics

Lesson 49 of 66 ยท Production Features

Messaging and Events Between Services

Send and receive messages in Spring Boot with Kafka and RabbitMQ, understand at-least-once delivery, write idempotent consumers and use the outbox pattern.

A direct HTTP call couples two services in time: if the receiver is down or slow, the sender suffers. A message broker sits in the middle. The sender hands a message to the broker and moves on; the receiver picks it up when it can. That decoupling is the reason order systems, notification pipelines and data integrations are usually built on messaging.

It also introduces a new class of problems. Messages can be delivered twice, arrive late, or arrive in surprising order, and "save to the database and publish a message" is not one atomic action. This lesson shows the Spring Boot code for Kafka and RabbitMQ (read-only, because no broker runs on this site), then explains the delivery guarantees and the two patterns that make messaging safe: idempotent consumers and the outbox. Both patterns are demonstrated in runnable stand-ins.

Why a broker in the middle

Time decoupling The receiver can be down for a while.
Load smoothing A spike queues up instead of crashing.
Fan-out One event, many independent consumers.

The price is eventual consistency and duplicate delivery.

1 / 2

Kafka and RabbitMQ in one minute

The two brokers you meet most often work differently.

  • Apache Kafka is a distributed log. Messages are appended to a topic, split into partitions, and kept for a retention period whether or not anyone read them. Consumers track their own position (the offset). Messages with the same key go to the same partition, which gives ordering per key. Consumers in one consumer group share the partitions, so adding consumers scales reading. Kafka is the usual choice for high throughput, event streams and replay.
  • RabbitMQ is a classic queue broker. Producers publish to an exchange, which routes by routing key into queues; a message is removed once a consumer acknowledges it. It is a good fit for task distribution and flexible routing, with richer per-message features (priorities, delayed or dead-lettered messages).

Choose by the workload, not fashion: replayable event history and huge volume point to Kafka; job queues and routing flexibility point to RabbitMQ. The Spring programming model is similar for both.

Kafka in Spring Boot

Add spring-boot-starter-kafka (Boot 4) or spring-kafka (Boot 3 and earlier), then configure the connection:

PROPERTIES
spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=shipping
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.producer.acks=all

Sending uses KafkaTemplate; receiving is one annotation:

Java
public record OrderPlaced(String eventId, long orderId, String email) {}

@Service
class OrderPublisher {
    private final KafkaTemplate<String, OrderPlaced> kafka;

    OrderPublisher(KafkaTemplate<String, OrderPlaced> kafka) {
        this.kafka = kafka;
    }

    void publish(OrderPlaced event) {
        // key = order id: all events of one order land in the same partition, in order
        kafka.send("orders.placed", String.valueOf(event.orderId()), event);
    }
}

@Component
class ShippingListener {

    @KafkaListener(topics = "orders.placed", groupId = "shipping")
    void on(OrderPlaced event) {
        // create the shipment
    }
}

send returns a future; the message is only safe once the broker has acknowledged it (acks=all waits for all in-sync replicas). Payloads are converted by serializers, commonly JSON (spring.kafka.producer.value-serializer, spring.kafka.consumer.value-deserializer and trusted packages must be configured, and a malformed message that cannot be deserialized needs an error-handling deserializer or it blocks the partition).

RabbitMQ in Spring Boot

Add spring-boot-starter-amqp:

PROPERTIES
spring.rabbitmq.host=localhost
spring.rabbitmq.username=shop
spring.rabbitmq.password=${RABBIT_PASSWORD}
Java
@Configuration
class RabbitConfig {
    @Bean Queue shippingQueue() { return new Queue("shipping.orders", true); }          // durable
    @Bean MessageConverter jsonConverter() { return new Jackson2JsonMessageConverter(); } // JacksonJsonMessageConverter in Boot 4
}

@Service
class OrderPublisher {
    private final RabbitTemplate rabbit;
    OrderPublisher(RabbitTemplate rabbit) { this.rabbit = rabbit; }

    void publish(OrderPlaced event) {
        rabbit.convertAndSend("", "shipping.orders", event);   // default exchange routes by queue name
    }
}

@Component
class ShippingListener {
    @RabbitListener(queues = "shipping.orders")
    void on(OrderPlaced event) { /* create the shipment */ }
}

By default the container acknowledges a message automatically after the listener returns normally. If the listener throws, the message is rejected and, depending on configuration, requeued, retried with backoff, or sent to a dead-letter queue. The converter class name varies by Boot version (Boot 4 moves to Jackson 3), so check the one your version provides. Both Spring modules also support transactions, batch listeners and manual acknowledgement for special cases.

Delivery guarantees: at-least-once is the realistic default

When the consumer finishes the work and then tells the broker "done" (an acknowledgement, or a committed offset in Kafka), a crash between the two steps means the broker delivers the same message again. That is at-least-once delivery: nothing is lost, but duplicates happen.

The opposite order, acknowledging first and processing afterwards (at-most-once), never duplicates but loses the message when the crash falls in between. For business events, losing data is almost always worse than handling it twice, so you pick at-least-once and make duplicates harmless.

Exactly-once across a broker and your database is, strictly speaking, not available. Kafka has transactional features that give exactly-once inside Kafka (read a topic, write another topic), but as soon as a side effect leaves the broker (a database row, an email, an HTTP call), you are back to at-least-once plus idempotence. The practical formula is:

at-least-once delivery + idempotent processing = effectively exactly-once

sequenceDiagram
    participant B as Broker
    participant C as Consumer
    participant D as Database
    B->>C: deliver message m1
    C->>D: apply effect
    Note over C: crash before ack
    B->>C: deliver m1 again (redelivery)
    C->>D: apply effect again?
    Note over C,D: an idempotent consumer detects the repeat and skips it

A crash between the effect and the acknowledgement causes redelivery. Safety comes from the consumer, not the broker.

Idempotent consumers

A handler is idempotent if running it twice has the same result as running it once. Some operations are naturally idempotent ("set status to SHIPPED"); others are not ("add 10 loyalty points", "send an email", "charge a card"). For those, record which message ids you already handled and ignore repeats:

Java
@Transactional
void handle(OrderPlaced event) {
    // processed_event has a UNIQUE constraint on event_id
    if (!processedEvents.markProcessed(event.eventId())) {
        return;                                // duplicate: skip
    }
    loyalty.addPoints(event.email(), 10);      // same transaction as the marker
}

The marker and the effect must commit together, in the same database transaction. If they were separate, a crash between them would reintroduce the problem. For effects outside your database (an email provider, a payment API) use their idempotency keys when offered, and accept the small residual risk otherwise.

Every message therefore needs a stable unique id (eventId) created by the producer when the event is born, not a broker offset that changes if you republish.

An in-memory broker with a crash

This stand-in models a one-partition topic, a consumer group offset and the "process, then commit" order. The consumer crashes once after applying the effect but before committing. A naive consumer applies the message twice; an idempotent one does not. What is real: the redelivery logic and the marker idea. What is the stand-in: no network, partitions, rebalancing or real offsets, and the "database" is a map.

Java Redelivery after a crash, naive vs idempotent

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Look at grace: the naive consumer gave her 20 points because m2 was redelivered after the crash, while the idempotent consumer recognized the id and gave 10. Nothing was lost in either case; only one of them stayed correct.

The dual-write problem and the outbox pattern

Consider the usual order flow: save the order, then publish OrderPlaced.

Java
@Transactional
void place(Order order) {
    orders.save(order);
    kafka.send("orders.placed", ...);   // not part of the database transaction!
}

The database and the broker are two systems, and no single transaction covers both. If the commit succeeds but the publish fails (broker down, crash), the order exists and nobody learns about it. If you publish first and the commit then fails, the world hears about an order that does not exist. Neither order of the two steps is safe. This is the dual-write problem.

The transactional outbox removes it by writing the event to an outbox table in the same database transaction as the business change. A separate relay reads unsent rows, publishes them to the broker and marks them as sent:

flowchart LR
    A["Service: one DB transaction"] --> B[("orders table")]
    A --> C[("outbox table")]
    C --> D["Relay (poller or CDC)"]
    D --> E["Broker"]
    E --> F["Consumers (idempotent)"]

The order and its event commit together; the relay delivers the event later, at least once.

Now both facts are true or false together, because they are in one transaction. The relay is also at-least-once (it can publish and crash before marking the row as sent), which is why consumers must be idempotent: the pattern and the idempotent consumer are two halves of one design. The relay can be a scheduled poller (@Scheduled, see Scheduling and Asynchronous Work), or change-data-capture tooling such as Debezium that reads the database log. Housekeeping matters too: delete or archive old outbox rows, and keep an index on the "unsent" rows so the poll stays cheap.

The second TryIt uses a real H2 database for the transaction and an in-memory list as the "broker". It places an order (order row and outbox row in one transaction), shows that a failing transaction leaves neither, then runs a relay that crashes after publishing, so the event is published twice and the idempotent consumer drops the duplicate. What is real: JDBC transactions and the pattern. What is the stand-in: the broker and the relay loop.

Java Transactional outbox on H2

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The duplicate on the broker is not a bug in the design; it is the expected price of at-least-once, absorbed by the consumer.

Practical guidance

  • Keep messages small and self-describing: an id, a type, a version and the data consumers need (or an id they can look up). Messages outlive code, so evolve them compatibly: add fields, never reuse or repurpose old ones.
  • Plan for poison messages. A message that always fails (bad data) must not block a partition or loop forever. Use retries with backoff and then a dead-letter topic or queue (DefaultErrorHandler with DeadLetterPublishingRecoverer for Kafka; dead-letter exchanges for RabbitMQ), and alert on its size.
  • Do not assume global order. Kafka orders per partition (use a key), RabbitMQ per queue, and retries can reorder things. Make handlers tolerant ("ignore an older version").
  • Observe it. Track consumer lag, processing time and failure counts (Observability); a growing lag is the earliest sign of trouble.
  • Test with a real broker (Testcontainers, see Integration Tests with Testcontainers), because mocks hide serialization, ordering and acknowledgement behavior.

For in-process reactions that do not need a broker, plain application events are simpler (Application Events and Runners); for the Jakarta view of messaging see JMS Messaging.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A consumer processes a message and crashes before acknowledging it. What does an at-least-once system do?

    Without the acknowledgement the broker cannot know the work finished, so it redelivers. That is why duplicates must be expected.
  2. 2. Why must the "already processed" marker be written in the same transaction as the business effect?

    Together they commit or roll back as one unit. Separately, a crash between them would reintroduce either duplicate effects or lost effects.
  3. 3. What problem does the transactional outbox solve?

    By storing the event in the same database transaction as the data change and relaying it afterwards, the change and its event cannot disagree.
  4. 4. Which statement about "exactly-once" delivery is the most accurate for services with databases and external effects?

    Once an effect leaves the broker, you cannot make delivery exactly-once. Duplicate-safe processing gives the same outcome.
  5. 5. In Kafka, what does using the order id as the message key give you?

    The key is hashed to pick a partition, and Kafka preserves order within a partition, so events of one order stay in sequence.

Summary

  • A broker decouples services in time and load; Kafka is a partitioned replayable log, RabbitMQ a routing queue broker. Spring Boot offers KafkaTemplate/@KafkaListener and RabbitTemplate/@RabbitListener.
  • Acknowledge after processing: delivery is at-least-once, so duplicates happen after crashes and rebalances.
  • Make consumers idempotent with a unique event id recorded in the same transaction as the effect.
  • Saving data and publishing a message is a dual write; the outbox pattern writes the event in the same transaction and relays it later.
  • Use retries, dead-letter destinations, versioned messages and lag monitoring, and test against a real broker.

Next: Resilience: Timeouts, Retries and Circuit Breakers.