Skip to content
Tutorials Center
My learning

Topics

Lesson 39 of 39 ยท Practice and Review

Kafka Cheat Sheet

A compact Apache Kafka cheat sheet with CLI commands, producer, consumer, topic and broker settings, delivery recipes and decision tables for quick lookup.

This page puts the commands, settings and rules of thumb from the whole Kafka course in one place. Use it when you are at a terminal, reviewing a configuration or preparing for an interview. Each section links to the lesson that explains the topic properly, so you can dig deeper when a line is not enough.

Commands and settings are read-only here, because the site cannot run a Kafka cluster. Two tiny stand-ins at the end (JavaScript calculators for partition counts and disk size) run in the editor. Default values change between Kafka versions, so treat the defaults below as typical and check the documentation for the version you run.

What is on this page

CLI commands Topics, consoles, groups, configs.
Settings Producer, consumer, topic, broker.
Recipes No loss, no duplicates, low latency.
Calculators Partitions and disk, runnable.
1 / 2

Command line tools

The scripts live in the bin folder of a Kafka download (kafka-topics.sh on Linux and macOS, bin\windows\*.bat on Windows). Replace localhost:9092 with your bootstrap server. See Your First Topic with the CLI.

Start a KRaft broker (single node, development)

Terminal
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
bin/kafka-storage.sh format -t "$KAFKA_CLUSTER_ID" -c config/server.properties   # config path depends on version
bin/kafka-server-start.sh config/server.properties

Topics

Terminal
bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic orders --partitions 3 --replication-factor 1
bin/kafka-topics.sh --bootstrap-server localhost:9092 --list
bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic orders
bin/kafka-topics.sh --bootstrap-server localhost:9092 --alter --topic orders --partitions 6   # can only increase
bin/kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic orders
bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --under-replicated-partitions

Console producer and consumer

Terminal
bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic orders \
  --property parse.key=true --property key.separator=:        # type  ana:order-1

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders --from-beginning \
  --property print.key=true --property print.partition=true --property print.offset=true

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders \
  --group audit --max-messages 10

Consumer groups and offsets

Terminal
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group fulfilment   # offsets and lag
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group fulfilment --members --verbose

# Reset offsets (the group must have no active members). Dry run first!
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group fulfilment --topic orders \
  --reset-offsets --to-earliest --dry-run
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group fulfilment --topic orders \
  --reset-offsets --to-datetime 2026-01-15T00:00:00.000 --execute

bin/kafka-get-offsets.sh --bootstrap-server localhost:9092 --topic orders --time -1   # latest offsets
bin/kafka-get-offsets.sh --bootstrap-server localhost:9092 --topic orders --time -2   # earliest offsets

See Offsets and Commits and Monitoring, Lag and Troubleshooting.

Configuration, ACLs, quorum, inspection

Terminal
bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type topics --entity-name orders \
  --alter --add-config retention.ms=604800000,min.insync.replicas=2
bin/kafka-configs.sh --bootstrap-server localhost:9092 --entity-type topics --entity-name orders --describe

bin/kafka-acls.sh --bootstrap-server localhost:9092 --add --allow-principal User:alice \
  --operation Read --topic orders --group fulfilment

bin/kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --status
bin/kafka-log-dirs.sh --bootstrap-server localhost:9092 --describe --topic-list orders
bin/kafka-dump-log.sh --files /var/lib/kafka/orders-0/00000000000000000000.log --print-data-log

bin/kafka-producer-perf-test.sh --topic perf --num-records 1000000 --record-size 1000 \
  --throughput -1 --producer-props bootstrap.servers=localhost:9092 acks=all

Producer settings

See Writing a Producer and Batching, Compression and Idempotence.

Setting Typical value Purpose
acks all How many replicas must confirm: 0, 1 or all
enable.idempotence true No duplicates from retries (per partition and producer session)
delivery.timeout.ms 120000 Total time allowed to deliver, including retries
retries very large Retry transient errors; bounded by the delivery timeout
linger.ms 5 to 50 for throughput How long to wait to fill a batch
batch.size 16384 and up Maximum bytes per partition batch
compression.type lz4, zstd, snappy, gzip Compress whole batches
max.in.flight.requests.per.connection 5 or less Unacknowledged requests; at most 5 keeps order with idempotence
buffer.memory 33554432 Memory for records waiting to be sent
max.request.size 1048576 Largest request; must fit your biggest record
transactional.id unique per instance Enables transactions and fencing

Consumer settings

See Reading with a Consumer, Consumer Groups and Rebalancing and Offsets and Commits.

Setting Typical value Purpose
group.id required for groups Names the consumer group
enable.auto.commit false for at-least-once control Commit offsets automatically in the background
auto.offset.reset earliest or latest Where to start when no committed offset exists (or it is out of range)
max.poll.records 500 Records returned per poll
max.poll.interval.ms 300000 Maximum time between polls before the member is removed
session.timeout.ms 45000 Time without heartbeats before the member is declared dead
heartbeat.interval.ms 3000 Heartbeat frequency (about a third of the session timeout)
fetch.min.bytes, fetch.max.wait.ms 1, 500 Trade latency for bigger fetches
isolation.level read_committed with transactions Hide uncommitted transactional records
partition.assignment.strategy cooperative-sticky How partitions are divided
group.instance.id stable per instance Static membership, avoids rebalances on quick restarts

Topic settings

See Retention, Segments and Disk Layout and Log Compaction and Tombstones.

Setting Typical value Purpose
cleanup.policy delete, compact or both Remove by age and size, or keep the newest record per key
retention.ms 604800000 (7 days) Maximum age of data before whole segments are deleted
retention.bytes -1 (unlimited) Maximum size per partition
segment.bytes 1073741824 Segment file size before rolling
min.insync.replicas 2 with RF 3 Minimum in-sync replicas for acks=all writes
max.message.bytes 1048588 Largest record batch the broker accepts for the topic
delete.retention.ms 86400000 How long tombstones stay on a compacted topic
min.cleanable.dirty.ratio 0.5 When the cleaner starts on a compacted topic

Broker settings

See Running a Cluster: KRaft, Brokers and Controllers and Security and Quotas.

Setting Purpose
process.roles broker, controller or both
node.id Unique id of this node
controller.quorum.voters The controller quorum (id@host:port,...)
listeners, advertised.listeners Where the broker binds and the address clients are told to use
log.dirs Directories for partition data
num.partitions, default.replication.factor Defaults for auto-created topics
unclean.leader.election.enable Keep false unless availability matters more than data
auto.create.topics.enable Turn off in production, create topics deliberately
message.max.bytes Largest batch accepted broker-wide

Delivery recipes

Goal Producer Consumer
No loss acks=all, idempotence, retries; topic RF 3 and min.insync.replicas=2 Commit after processing
No duplicates from retries enable.idempotence=true Idempotent handler for crash replays
Exactly-once inside Kafka transactional.id, transactions isolation.level=read_committed; consume-transform-produce with offsets in the transaction
Lowest latency linger.ms=0, acks=1 (accepting risk) Small fetch.min.bytes
Highest throughput linger.ms 20+, large batches, compression.type=lz4 or zstd Larger fetches and max.poll.records
Replay history n/a New group id or reset offsets, retention must still hold the data

See At-Most-Once, At-Least-Once, Exactly-Once and Transactions and Exactly-Once.

Decision tables

Question Answer
Need order for related events? Same key, so same partition. Order exists only inside a partition.
Need more consumer parallelism? More partitions (up to your target), then more consumers, never more consumers than partitions.
Need only the latest value per key? cleanup.policy=compact.
Need an audit trail? Retention-based topic, not compaction.
Message keeps failing? Bounded retries, then a dead letter topic. See Retries, Dead Letter Topics and Poison Messages.
Format will change? Schema registry with a compatibility mode. See Schemas, Avro, Protobuf and JSON.
Moving data to or from a database? Kafka Connect. See Kafka Connect.
Aggregations, joins, windows? Kafka Streams or a similar processor. See Kafka Streams and Stream Processing.

Troubleshooting quick list

Symptom First checks
Lag growing One partition or all? Handler time, consumer count versus partitions, rebalances.
Constant rebalances max.poll.interval.ms, slow processing in the poll loop, session timeout, unstable members.
Producer errors "not enough replicas" Brokers down, ISR below min.insync.replicas.
Record too large max.request.size, message.max.bytes, max.message.bytes, consumer fetch sizes.
Consumer starts at the wrong place auto.offset.reset, committed offsets, new group.id.
Duplicates Producer retries without idempotence, crash before commit.

Two small calculators

The next two stand-ins turn the rules of thumb into numbers. Change the inputs and run them again, but measure your own rates on real hardware before relying on them.

JS Estimate the partition count

Press Run (or Ctrl + Enter) to see the console output here.

No output.

A slow handler, not Kafka, drove the number from 9 to 36: the consuming side is often the bottleneck.

JS Estimate disk for a retention period

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The total is spread over all brokers' disks, and compression usually shrinks it. Retention is a cost lever: one day instead of seven cuts the disk need by a factor of seven.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which command shows the lag of the consumer group fulfilment?

    kafka-consumer-groups.sh --describe lists the committed offset, log end offset and lag per partition for the group.
  2. 2. Which settings together give durable writes on a topic with replication factor 3?

    With acks=all and at least two in-sync replicas required, an acknowledged record is on at least two brokers.
  3. 3. You need the newest value per key, such as a user profile table. Which topic setting fits?

    Compaction keeps the latest record per key, so the topic can be read as a table.
  4. 4. Before resetting a group's offsets with --execute, what should you do?

    A dry run shows what would change, and the reset only works for an inactive group.

Summary

  • The kafka-topics, kafka-console-*, kafka-consumer-groups and kafka-configs scripts cover most daily work; use --dry-run before resetting offsets.
  • Producers: acks=all, idempotence, sensible linger.ms and compression. Consumers: control commits, tune poll settings.
  • Topics: choose delete or compact, set retention and min.insync.replicas deliberately.
  • Order is per partition; parallelism is limited by partitions.
  • Defaults vary by version, so check the documentation of the release you run.

That completes the course. Revisit What Is Apache Kafka? any time to review the big picture.