Skip to content
Tutorials Center
My learning

Topics

Lesson 2 of 39 ยท Getting Started

Kafka vs RabbitMQ, ActiveMQ, Pulsar and Cloud Queues

Compare Kafka with RabbitMQ, ActiveMQ, Apache Pulsar, Amazon SQS, Google Pub/Sub and Azure Service Bus: queues versus logs, push versus pull, replay, routing and when to choose each.

"Should we use Kafka?" is really two questions: do we need a log of events or a queue of jobs, and who will run it? Kafka is excellent at the first and merely adequate at the second. Other systems make the opposite trade. This lesson gives you the vocabulary to compare them honestly, so you can pick by requirement instead of by fashion.

Real versus stand-in. This lesson compares real products in general terms; features and limits change between versions, so check current documentation before deciding. The TryIt blocks are plain JavaScript models of the two styles, a queue and a log, to show the behavioral difference. They are not any product.

Two different ideas

A message queue holds work to be done. A producer adds a message, one consumer takes it, processes it and acknowledges it, and the broker then deletes it. The queue's natural state is empty.

An event log (Kafka's model, see The Commit Log Model) holds history. Records are appended and kept for a retention period no matter who has read them. Each consumer group has its own position, so many independent readers can read the same data, and any of them can go back and read it again.

Queue or log?

Queue Take a message, process it, it is gone.
Log Append events, keep them, readers track a position.
Replay Only a log lets you read the past again.

Most differences below follow from this one choice.

1 / 2
JS A queue versus a log

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Push versus pull

Classic brokers such as RabbitMQ push messages to consumers, up to a prefetch limit, and track acknowledgements per message. Kafka consumers pull batches at their own pace and track only an offset (see Offsets and Commits). Pull gives natural backpressure and big efficient batches. Per-message acknowledgement gives fine control: you can reject one message, requeue it, delay it or give it a priority, none of which a plain Kafka partition offers. A single slow message in Kafka blocks the records behind it in the same partition.

Routing

RabbitMQ and ActiveMQ have rich routing: exchanges, binding keys, topic patterns and headers decide which queues receive a message. In Kafka the producer simply names a topic and, optionally, a key; consumers choose what to subscribe to. Complex routing is done in code or by stream processing (Kafka Streams), not in the broker.

The main contenders

System Model Good at Watch out for
Apache Kafka Partitioned, replicated log; pull High throughput, replay, many consumers, stream processing, event sourcing No per-message priority, delay or routing; operating a cluster takes skill
RabbitMQ Broker with exchanges and queues; push (also has Streams) Task queues, flexible routing, request/reply, low latency per message Throughput and retention are not its strength; replay needs its Streams feature
ActiveMQ (Classic / Artemis) JMS-compliant broker Java enterprise integration, JMS, many protocols Smaller ecosystem for streaming
Apache Pulsar Log with separate compute (brokers) and storage (BookKeeper) Multi-tenancy, geo-replication built in, queue and stream semantics in one More moving parts; smaller community than Kafka
Amazon SQS Managed queue Zero ops, simple decoupling, scales automatically No replay, limited ordering (FIFO queues have lower throughput)
Google Pub/Sub Managed pub/sub Global, serverless, push or pull subscriptions Less control over partitions and offsets
Azure Service Bus / Event Hubs Service Bus: queues and topics; Event Hubs: log with a Kafka-compatible endpoint Azure integration Two products for two models

Kafka-compatible services (Confluent Cloud, Amazon MSK, Redpanda, Event Hubs' Kafka endpoint) let you use Kafka clients without running brokers yourself, which often removes the biggest objection to Kafka.

A decision helper

JS Which broker fits these needs?

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The helper is deliberately crude: real decisions weigh team skills, existing infrastructure, latency targets and budget too.

When Kafka is the wrong tool

  • You need one-off background jobs with retries, delays and priorities: a task queue is simpler.
  • You have a handful of messages per minute and nobody to operate a cluster: a managed queue costs less effort.
  • You need a request/reply call with a response: use HTTP or gRPC.
  • You need to query data by arbitrary fields: Kafka is not a database (see What Is Apache Kafka?).

Many systems use both: Kafka as the backbone of events, and a queue for job processing downstream.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the key behavioral difference between a classic queue and Kafka's log?

    Retention and per-group positions are what allow replay and multiple independent consumers.
  2. 2. Which need points toward RabbitMQ over Kafka?

    Per-message acknowledgement, priorities, delays and exchange routing are core RabbitMQ features.
  3. 3. What do consumers do in Kafka?

    Kafka consumers poll for records and commit offsets; the broker does not track each message.
  4. 4. Which is a managed queue with no replay?

    SQS is a simple managed queue; messages are removed after processing.

Summary

  • Queues hold work and delete it after processing; logs keep history and let many groups read and replay.
  • Kafka pulls in batches and tracks offsets; RabbitMQ-style brokers push and acknowledge each message.
  • RabbitMQ and ActiveMQ lead on routing, priorities and delays; Kafka and Pulsar lead on throughput and replay.
  • Managed queues (SQS, Pub/Sub, Service Bus) remove operations work; managed Kafka does the same for the log model.
  • Choose from requirements: replay, fan-out, throughput, routing and who operates it.

Next: try it yourself with First Steps with the Kafka CLI.