Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 36 ยท Reactive and Messaging

Reactive Messaging with Kafka and AMQP

Build event-driven Quarkus services with SmallRye Reactive Messaging: @Incoming, @Outgoing, Emitter, channels, Kafka configuration, acknowledgment and dead letters.

HTTP is request and response: a caller waits for the answer. Many jobs do not fit that shape. An order is placed, and later the warehouse, the invoice system and the email sender must each react, without the shop blocking on any of them. For this you publish an event to a broker such as Apache Kafka, RabbitMQ (AMQP) or Pulsar, and other services consume it when they are ready.

Quarkus ships SmallRye Reactive Messaging for this. You write plain methods annotated with @Incoming and @Outgoing, and configuration decides whether a channel is Kafka, AMQP, MQTT or just in memory. This lesson covers channels, the connector configuration, sending from imperative code with Emitter, acknowledgment and failure handling. It uses Mutiny types and follows the thread rules from the event loop lesson. For Kafka basics see the Kafka tutorial.

The broker and the runtime are not available on this site, so the real code is read-only and the TryIts implement the same ideas in plain Java.

Channels, not topics

Your code talks to channels: named streams inside the application. A connector attaches a channel to the outside world. The same method works against Kafka in production and an in-memory channel in a test.

Java
import org.eclipse.microprofile.reactive.messaging.Incoming;
import org.eclipse.microprofile.reactive.messaging.Outgoing;

@ApplicationScoped
public class OrderPipeline {

    @Incoming("orders")                 // consumes from channel "orders"
    @Outgoing("invoices")               // publishes the return value to "invoices"
    public Invoice toInvoice(Order order) {
        return new Invoice(order.id(), order.totalCents() * 119 / 100);
    }

    @Incoming("invoices")
    public void archive(Invoice invoice) {
        Log.infof("invoice %s: %d", invoice.orderId(), invoice.amountCents());
    }
}
PROPERTIES
# Kafka for "orders" (incoming) ...
mp.messaging.incoming.orders.connector=smallrye-kafka
mp.messaging.incoming.orders.topic=shop-orders
mp.messaging.incoming.orders.group.id=invoice-service
mp.messaging.incoming.orders.auto.offset.reset=earliest
mp.messaging.incoming.orders.value.deserializer=io.quarkus.kafka.client.serialization.ObjectMapperDeserializer

# ... "invoices" stays inside the app (no connector): methods are wired directly

Two details worth learning at once:

  • A channel without a connector is an internal channel; Quarkus wires an @Outgoing("x") method straight to an @Incoming("x") method in the same application.
  • The Kafka extension is quarkus-messaging-kafka (older name quarkus-smallrye-reactive-messaging-kafka). In dev mode and in tests Dev Services starts a Kafka broker in a container for you when Docker is available and no bootstrap.servers is configured. See dev mode and Dev Services.

Messages flow through channels

Source A Kafka topic or an Emitter.
Processor @Incoming plus @Outgoing.
Sink A method that consumes.

The connector decides what is outside the application.

1 / 2

Sending from a REST endpoint

A REST method is not part of a stream, so use an Emitter to push into a channel:

Java
import org.eclipse.microprofile.reactive.messaging.Channel;
import org.eclipse.microprofile.reactive.messaging.Emitter;

@Path("/orders")
public class OrderResource {

    @Channel("orders-out") Emitter<Order> emitter;

    @POST
    public Response place(Order order) {
        emitter.send(order);
        return Response.accepted().build();
    }
}
PROPERTIES
mp.messaging.outgoing.orders-out.connector=smallrye-kafka
mp.messaging.outgoing.orders-out.topic=shop-orders
mp.messaging.outgoing.orders-out.value.serializer=io.quarkus.kafka.client.serialization.ObjectMapperSerializer

emitter.send(payload) returns a CompletionStage<Void> that completes when the broker acknowledges. Ignoring it is fine for fire-and-forget; wait on it when the HTTP answer should depend on it. If the application sends faster than the broker accepts, the emitter's buffer fills up and an error appears; add @OnOverflow(OnOverflow.Strategy.BUFFER) with a size, or slow the producer down.

Java Stand-in: channels, a processor and an emitter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Acknowledgment: when is a message "done"?

A broker needs to know a message was processed, otherwise it would send it again after a restart. Reactive Messaging calls this acknowledgment. For payload methods (Invoice toInvoice(Order o)) the default is automatic: the message is acknowledged when your method returns normally, and negatively acknowledged when it throws.

Kafka's default failure strategy for a nack is fail: the application is marked unhealthy and stops consuming, which is the safest choice because it never skips data silently. You can choose another:

PROPERTIES
mp.messaging.incoming.orders.failure-strategy=dead-letter-queue
mp.messaging.incoming.orders.dead-letter-queue.topic=shop-orders-dlq
# or: ignore (log and continue)
Strategy Effect of a failed message
fail Stop the channel, report unhealthy (default)
ignore Log it and continue with the next
dead-letter-queue Copy the record to another topic and continue

For full control, take a Message<T> instead of the bare payload and call ack() or nack(throwable) yourself.

At-least-once delivery means idempotent handlers

Kafka consumers commit their position after processing. If the application crashes between "processed" and "committed", the message comes again after restart. So handlers must tolerate duplicates: store the order id and skip if it was already handled, or make the write itself idempotent (an upsert). This is the single most important habit in event-driven code.

Java Stand-in: nack, dead letters and idempotent handling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Other connectors and useful pieces

  • AMQP 1.0 and RabbitMQ: quarkus-messaging-amqp or quarkus-messaging-rabbitmq, with connector=smallrye-amqp or smallrye-rabbitmq.
  • In-memory (smallrye-reactive-messaging-in-memory): swap a Kafka channel for a testable one in @QuarkusTest, covered in mocks and test profiles.
  • Kafka Streams (quarkus-kafka-streams) for stateful aggregation and joins, and Apicurio Registry for Avro or JSON schemas.
  • Metrics and health: the Kafka connector contributes readiness and liveness checks (see health and metrics).
  • Blocking handlers: a method annotated @Blocking runs on a worker, so JDBC is allowed. Without it, @Incoming methods on Kafka run on the I/O thread.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is a channel in SmallRye Reactive Messaging?

    Channels are internal names. Configuration maps them to Kafka, AMQP or leaves them in memory.
  2. 2. How do you publish a message from a JAX-RS method?

    An Emitter bridges imperative code and a channel.
  3. 3. A payload-based @Incoming method throws an exception. What does Kafka do by default?

    The default failure strategy is fail, so no data is silently lost.
  4. 4. Why should message handlers be idempotent?

    Offsets are committed after processing, so a message can be redelivered. Handling it twice must be harmless.

Summary

  • Code uses channels with @Incoming, @Outgoing and Emitter; configuration attaches connectors.
  • Dev Services starts a Kafka broker automatically in dev and test.
  • Failure strategies: fail, ignore, dead-letter-queue.
  • Delivery is at-least-once, so make handlers idempotent.

Next: Dev Mode and Dev Services