Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 37 ยท Networking and Servers

MQTT: Messaging Between Pis and Devices

Connect many sensors and Pis with MQTT: topics, wildcards, QoS levels, retained messages, Mosquitto and paho-mqtt, with a runnable publish-subscribe broker model.

When one Pi reads a sensor and another needs the value, you could build an HTTP API on each. It works, but it gets messy when there are ten sensors, three displays and a phone app. MQTT is a lightweight messaging protocol made for this. Devices connect to a central broker, publish messages to named topics, and others subscribe to the topics they care about. The sender and receiver never talk to each other directly.

A Pi runs the broker well (Mosquitto uses a few megabytes of RAM), and tiny ESP32 boards and phones speak MQTT too, which is why it is the glue of home automation. Mosquitto cannot run on this website, so broker configuration and paho-mqtt code are read-only, and the runnable blocks are a small in-memory model of a broker.

The publish-subscribe model

flowchart LR
    S1["Greenhouse Pi<br/>publishes"] -->|"home/greenhouse/temp"| B["Broker"]
    S2["Garage ESP32<br/>publishes"] -->|"home/garage/door"| B
    B -->|"home/+/temp"| D1["Dashboard"]
    B -->|"home/#"| D2["Logger Pi"]

Publishers and subscribers only know the broker and the topic names.

A topic is a path-like string such as home/greenhouse/temperature. Levels are separated by /. A message is just bytes (usually text or JSON) sent to a topic. Two wildcards work in subscriptions only:

  • + matches exactly one level: home/+/temperature matches home/greenhouse/temperature and home/garage/temperature.
  • # matches any number of remaining levels and must be last: home/# matches everything below home.

Installing Mosquitto

Terminal
sudo apt install mosquitto mosquitto-clients
sudo systemctl enable --now mosquitto
mosquitto_sub -h localhost -t "home/#" -v          # terminal 1: listen
mosquitto_pub -h localhost -t "home/test" -m "hi"  # terminal 2: send

The second command makes home/test hi appear in the first terminal. Since Mosquitto 2, it only listens on localhost until you configure a listener and users, a good default. A file in /etc/mosquitto/conf.d/local.conf:

TEXT
listener 1883
allow_anonymous false
password_file /etc/mosquitto/passwd

Create the user with sudo mosquitto_passwd -c /etc/mosquitto/passwd ana, restart the service, and pass -u ana -P secret to the clients.

A Python publisher and subscriber

With sudo apt install python3-paho-mqtt, a publisher is short:

Python
import json, time
import paho.mqtt.client as mqtt

client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
client.username_pw_set("ana", "secret")
client.connect("pi-hub.local", 1883)
client.loop_start()
while True:
    client.publish("home/greenhouse/temperature",
                   json.dumps({"c": 22.4}), qos=1, retain=True)
    time.sleep(30)

A subscriber sets an on_message callback that receives each message:

Python
def on_message(client, userdata, msg):
    print(msg.topic, msg.payload.decode())

client.on_message = on_message
client.subscribe("home/+/temperature")
client.loop_forever()

Modelling a broker

The stand-in below is a real, working miniature: it stores subscriptions and delivers a published message to every subscriber whose pattern matches, with the + and # rules.

Python Stand-in: a tiny in-memory MQTT broker

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The temperature message reaches both subscribers; the door message reaches only the logger.

Quality of service and retained messages

MQTT lets you choose how hard it tries to deliver:

QoS Meaning Use for
0 At most once, fire and forget Frequent readings where a loss is fine
1 At least once, may duplicate Most sensor data
2 Exactly once, slowest Commands that must not repeat

A retained message is stored by the broker and delivered immediately to any new subscriber. It solves "what is the temperature right now?" for a dashboard that has just started. A last will message, set at connect time, is published by the broker if a device vanishes without disconnecting, so home/greenhouse/status can switch from online to offline automatically.

Python Stand-in: retained messages for late subscribers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Design tips

  • Use a clear hierarchy (site/room/device/measurement) and lowercase names.
  • Do not put spaces or a leading / in topics.
  • Send JSON for several values; send plain numbers for one.
  • Secure the broker with passwords, and TLS (port 8883) if it leaves your network.
  • Keep payloads small; MQTT is built for tiny messages.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which subscription matches home/garage/door and home/kitchen/door but not home/garage/light?

    • matches one level, so home/+/door fits both doors only.
  2. 2. What does a retained message do?

    The broker keeps the most recent retained message per topic.
  3. 3. Which QoS level guarantees exactly-once delivery?

    QoS 2 uses a four-step handshake to avoid duplicates.
  4. 4. Who needs to know the IP address of the subscriber when a sensor publishes?

    Publishers and subscribers only connect to the broker.

Summary

  • MQTT decouples devices: publishers send to topics, subscribers listen to patterns, the broker routes.
  • + matches one level and # matches the rest; both work only in subscriptions.
  • Mosquitto is the usual Pi broker; secure it with users and a listener config.
  • QoS 0, 1, 2 trade speed for delivery guarantees; retained and last-will messages help dashboards.
  • Plan a topic hierarchy early.

Next: Project: A Weather Station