Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 36 ยท MQTT and Messaging

Build a Toy MQTT Broker over TCP

Write a working publish/subscribe broker with real TCP sockets on loopback, a simple text protocol, wildcards and retained messages, then read what real MQTT adds.

The best way to understand a protocol is to build a small version. In this lesson you will write a broker and two clients in Node.js that talk over real TCP sockets on the loopback address (127.0.0.1), all inside one program so it runs in the editor. We use a readable line-based text protocol instead of MQTT's binary packets, but the logic (subscriptions, wildcards, retained messages, fan-out) is the same as in the lessons before.

This is a teaching toy, labelled as such. It is not MQTT on the wire, has no authentication and no QoS. Never use it for real devices; use Mosquitto, EMQX, HiveMQ or a cloud service.

The protocol

Each message is one line of text ending in a newline:

Line Meaning
SUB <filter> subscribe to a topic filter (+ and # allowed)
PUB <topic> <retain 0/1> <payload...> publish
MSG <topic> <payload...> broker to client: a delivered message

TCP is a byte stream, not a message stream: one data event can contain half a line or three lines. A parser must buffer bytes and split on newlines. This is the "framing" problem that every protocol solves (MQTT does it with a length field).

sequenceDiagram
  participant S as Sensor client
  participant B as Broker
  participant D as Dashboard client
  D->>B: SUB home/#
  S->>B: PUB home/temp 1 21.5
  B-->>D: MSG home/temp 21.5

The broker

Node.js A broker with TCP, wildcards and retained messages

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Read the output: the dashboard receives both live messages. The late phone connects after the door message and still receives it, because the broker stored it as retained and replays it when the SUB arrives. Notice that the late phone never sees the kitchen temperature, which was not retained and which its filter did not match anyway.

What real MQTT adds

Our toy leaves out most of what makes MQTT robust. Compare:

Feature Toy broker MQTT 3.1.1 / 5
Framing newline binary header with variable length field
Identity none client ID, username, password, TLS certificates
Reliability none QoS 0, 1, 2 with packet IDs and acknowledgments
Dead peer detection none keep-alive and PINGREQ
Offline devices none persistent sessions, queues
Failure notice none Last Will
Access control none ACLs per topic and client
Scale one process clustering, bridges, shared subscriptions

A real packet is compact. The publish of 21.5 to home/temp takes only about 20 bytes on the wire, which is why MQTT is friendly to cellular and battery devices. You can see the structure with a small hand-built packet:

Node.js Encode a real MQTT PUBLISH packet

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Libraries hide this, but knowing it explains why MQTT topics and payloads should be short.

Exercises

  1. Add a UNSUB <filter> command.
  2. Make the broker send MSG only once even if two filters of the same client match.
  3. Add a WILL command whose message is published when the socket closes without a BYE line.

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. Why does the toy broker buffer incoming data and split it on newlines?

    TCP has no message boundaries; the application protocol must define framing, here a newline.
  2. 2. Why does the late phone receive the door state?

    The message was published with retain flag 1, so the broker stored it.
  3. 3. Which of these does the toy broker NOT provide that real MQTT does?

    The toy has wildcards, fan-out and retain, but no acknowledgments or QoS.

Summary

  • A broker is a socket server plus a subscription list and a matching function.
  • TCP needs framing: buffer bytes and split into messages.
  • Retained messages are replayed when a matching subscription arrives.
  • Real MQTT adds compact binary framing, QoS, sessions, will, keep-alive, authentication and access control.

Next: IoT Data Pipelines: From Sensor to Insight