Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 36 ยท Projects and Practice

Project: A Live Metrics Dashboard

Build a live dashboard with Socket.IO: topic rooms, a snapshot on subscribe, throttled and batched updates, volatile events and per-client backpressure.

Dashboards are where real-time shines, and where naive designs fall over. If the server emits every change to every client, a busy system sends thousands of messages per second to browsers that can only paint 60 frames. This project builds a dashboard that stays smooth under load, using a few rules: clients subscribe to topics, new subscribers get a snapshot first, updates are batched and throttled on a timer, and slow clients are protected with volatile events.

Socket.IO is not installed in the runner on this site. The real server and client code is read-only. The runnable examples simulate the same logic with plain Node.js, using fake timers so the output is deterministic.

Requirements

  1. Several metric streams (topics): cpu, memory, orders.
  2. A client subscribes to the topics it displays, and only receives those.
  3. On subscribe, the client gets the current value immediately (no blank charts).
  4. Updates arrive at most a few times per second per topic, even if the source produces hundreds.
  5. A slow or busy client never makes the server buffer without limit.
  6. Optional authorization: some topics are for admins only.

Design

Data flow

Source Produces thousands of data points.
Aggregator Keeps the latest value or a batch per topic.
Timer Flushes every 250 ms to the topic room.
Browsers Draw at most once per message.

The timer, not the source, decides the pace.

1 / 2

Two shapes for the data decide the message format:

  • State (current CPU percentage): the latest value is all that matters. Send only the newest value each tick and drop the rest, a technique called conflation.
  • Events (new orders): every item matters. Send a batch of everything since the last tick.

Server

JavaScript
import { createServer } from "node:http";
import { Server } from "socket.io";

const httpServer = createServer();
const io = new Server(httpServer, { cors: { origin: process.env.CLIENT_ORIGIN } });

const TOPICS = { cpu: "state", memory: "state", orders: "events" };
const ADMIN_ONLY = new Set(["orders"]);

const latest = new Map();    // topic -> newest value (for state topics, and for snapshots)
const pending = new Map();   // topic -> items since last flush (for event topics)

// Called by your data sources, as often as they like
export function publish(topic, value) {
  if (TOPICS[topic] === "events") {
    pending.set(topic, [...(pending.get(topic) ?? []), value]);
  }
  latest.set(topic, value);
}

io.use((socket, next) => {
  socket.data.isAdmin = socket.handshake.auth?.role === "admin"; // in real life: verify a token
  next();
});

io.on("connection", (socket) => {
  socket.on("subscribe", (topic, ack) => {
    if (!(topic in TOPICS)) return ack?.({ ok: false, error: "unknown topic" });
    if (ADMIN_ONLY.has(topic) && !socket.data.isAdmin) return ack?.({ ok: false, error: "forbidden" });
    socket.join(`topic:${topic}`);
    ack?.({ ok: true, snapshot: latest.get(topic) ?? null });   // value NOW, not "wait for the next tick"
  });

  socket.on("unsubscribe", (topic) => socket.leave(`topic:${topic}`));
});

// Flush every 250 ms
setInterval(() => {
  for (const [topic, kind] of Object.entries(TOPICS)) {
    const room = io.to(`topic:${topic}`);
    if (kind === "state") {
      if (latest.has(topic)) room.volatile.emit("update", { topic, value: latest.get(topic), at: Date.now() });
    } else {
      const batch = pending.get(topic);
      if (batch?.length) {
        room.emit("batch", { topic, items: batch });   // events must not be dropped: not volatile
        pending.delete(topic);
      }
    }
  }
}, 250);

httpServer.listen(3000);

Why these choices:

  • One room per topic means the server does not loop over all sockets for each update, and clients get only what they asked for.
  • volatile for state updates: if a client's connection is busy or not ready, the newest value is simply skipped, and the next tick brings a fresher one anyway. There is no backlog.
  • The batch event is not volatile, because losing orders would be a bug. For critical streams, add an offset so a client can detect gaps (delivery guarantees).
  • The snapshot is returned in the acknowledgement, so the client has data in one round trip, in the same reply that confirms the subscription.

Client

JavaScript
import { io } from "socket.io-client";

const socket = io("http://localhost:3000", { auth: { role: "viewer" } });
const wanted = ["cpu", "memory"];

function subscribeAll() {
  for (const topic of wanted) {
    socket.timeout(3000).emit("subscribe", topic, (err, res) => {
      if (err || !res.ok) return showStatus(topic, err ? "timeout" : res.error);
      if (res.snapshot !== null) draw(topic, res.snapshot);
    });
  }
}

socket.on("connect", subscribeAll);          // also runs after every reconnect
socket.on("update", ({ topic, value }) => schedulePaint(topic, value));
socket.on("batch", ({ topic, items }) => items.forEach((i) => addRow(topic, i)));

// Paint at most once per animation frame, whatever arrives
const latestValue = new Map();
let scheduled = false;
function schedulePaint(topic, value) {
  latestValue.set(topic, value);
  if (scheduled) return;
  scheduled = true;
  requestAnimationFrame(() => {
    scheduled = false;
    for (const [t, v] of latestValue) draw(t, v);
    latestValue.clear();
  });
}

subscribeAll runs on every connect, because a reconnected socket is in no rooms. The client has its own throttle too: requestAnimationFrame batches many updates into one paint.

Simulation: conflation and batching

The first stand-in shows the core effect: a source producing hundreds of points per second, a server that flushes every 250 ms, and what a subscriber actually receives. Time is simulated.

Node.js Conflation and batching on a timer (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

A hundred CPU readings became four updates; thirty-three orders became four batches that still contain every order. Ben, who is not subscribed to orders, never saw them.

Simulation: snapshot and subscription control

The second stand-in covers subscribe/unsubscribe, authorization and the snapshot in the ack.

Node.js Subscriptions, snapshots and authorization (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The viewer is refused orders before any data leaves the server. Authorization happens at subscribe time, which is where room membership is decided (security hardening).

Scaling the dashboard

  • With several servers, each runs its own setInterval and needs the data. Feed all servers from a shared source: Redis pub/sub, a message queue, or the same database poll. The Redis adapter then delivers the topic rooms across nodes.
  • If aggregation is expensive, compute it once in a worker and publish the result; servers only fan out.
  • Cache the snapshot in Redis so a new server instance can answer subscribers immediately.
  • Cap per-client subscriptions to avoid a client subscribing to a thousand topics.
  • Measure messages per second out, per topic, in monitoring.

Extensions

  1. Add a history request that returns the last 60 points of a topic for the chart's initial draw.
  2. Let clients choose their own update rate (250 ms, 1 s), with one room per topic and rate.
  3. Detect stale data: emit a heartbeat with a timestamp and turn the widget grey when it stops.
  4. Replace polling in your source with the real system's events.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is conflation?

    For state data only the latest value matters, so intermediate values can be skipped.
  2. 2. Why are batches of orders not sent with volatile emit?

    Volatile means it is fine to lose the message. That suits repeated state values, not a list of events that must all arrive.
  3. 3. Why return the snapshot in the subscribe acknowledgement?

    Without a snapshot a new subscriber would see an empty chart until the next update.
  4. 4. Where should authorization for a topic be checked?

    Room membership decides who receives data, so the server must check permission before joining.

Summary

  • Use one room per topic and let clients subscribe; reply with a snapshot in the ack.
  • Throttle on a server timer: conflate state values, batch event items.
  • Use volatile for repeated state updates and normal emits for items that must arrive.
  • Re-subscribe on every connect and throttle painting with requestAnimationFrame.
  • Authorize subscriptions server-side and feed multiple servers from a shared source.

Next: Practice and Interview Questions