Skip to content
Tutorials Center
My learning

Topics

Lesson 70 of 133 ยท Data and Storage

MongoDB and Redis from Node.js

Use MongoDB and Redis from Node.js - official driver patterns, one client per process, caching with TTL and cache-aside, stampede control, key design, and runnable stand-ins for the core ideas.

Databases from Node.js gave you the map: relational databases for structured data, MongoDB for flexible documents, and Redis for fast shared state. This lesson goes deeper on the last two. They are the most common non-SQL pair in Node.js backends, and they are often used together: MongoDB as the main store, Redis in front of it as a cache and a place for short-lived data.

Our editor cannot install npm packages or run database servers, so the driver code in this lesson is read-only: it is what you would write on your own machine after npm install mongodb redis. To keep the lesson hands-on, each part is paired with a small runnable stand-in that implements the core idea in plain Node.js, so you can run, change and break it here.

How the pieces fit together

A typical request path with both databases looks like this. The cache answers most reads; the document store answers the rest and is the source of truth.

flowchart LR
    C["Client"] --> A["Node.js app"]
    A -->|"1. GET user:42"| R[("Redis")]
    R -->|"hit: return"| A
    A -->|"2. on miss: query"| M[("MongoDB")]
    M -->|"document"| A
    A -->|"3. SET user:42 with TTL"| R

Cache-aside: look in Redis first, fall back to MongoDB on a miss, then store the result in Redis with an expiry.

MongoDB: documents and the driver

MongoDB stores documents, JSON-like objects (stored as BSON), in collections. Documents in one collection do not have to share the same fields, which makes it comfortable for data whose shape changes. Every document has an _id; if you do not supply one, the driver creates an ObjectId.

The official driver is the mongodb package. Its API is promise-based. A first tour, read-only:

JavaScript
// read-only: requires "npm install mongodb" and a running MongoDB server
import { MongoClient, ObjectId } from 'mongodb';

const client = new MongoClient(process.env.MONGO_URL, {
  maxPoolSize: 20,                  // connections kept by this client
  serverSelectionTimeoutMS: 5_000,  // fail fast if no server is reachable
});
await client.connect();

const users = client.db('shop').collection('users');

// create
const { insertedId } = await users.insertOne({
  name: 'Ada',
  email: '[email protected]',
  tags: ['vip'],
  address: { city: 'London', country: 'UK' },
  createdAt: new Date(),
});

// read
const ada = await users.findOne({ _id: insertedId });
const londoners = await users
  .find({ 'address.city': 'London', tags: 'vip' })
  .project({ name: 1, email: 1 })    // only these fields (plus _id)
  .sort({ name: 1 })
  .limit(20)
  .toArray();

// update and delete
await users.updateOne({ _id: insertedId }, { $set: { 'address.city': 'Leeds' }, $inc: { logins: 1 } });
await users.deleteOne({ _id: new ObjectId('64b7f0c2a1b2c3d4e5f60718') });

await client.close();

A few things deserve attention:

  • Filters are objects. { 'address.city': 'London' } reaches into a nested document with dot notation. Matching a field against an array (tags: 'vip') means "the array contains this value".
  • Operators start with $. Query operators ($gt, $in, $or) go in filters; update operators ($set, $inc, $push) go in updates. An update without an operator would replace the whole document, which the driver rejects for updateOne.
  • Cursors are lazy. find() returns a cursor. Call .toArray() for small results, or iterate with for await (const doc of cursor) for large ones so you never hold millions of documents in memory.
  • Project and limit. Return only what you need. An unbounded find({}) on a large collection is the NoSQL version of SELECT *.
  • Create indexes. await users.createIndex({ email: 1 }, { unique: true }) makes lookups fast and enforces uniqueness. A query that scans the whole collection is slow in MongoDB just as in SQL. Use explain() to check.

Aggregation and transactions

Complex reports use the aggregation pipeline: an array of stages that transform documents step by step.

JavaScript
// read-only: count VIP users per country, largest first
const perCountry = await users.aggregate([
  { $match: { tags: 'vip' } },
  { $group: { _id: '$address.country', total: { $sum: 1 } } },
  { $sort: { total: -1 } },
]).toArray();

Single-document writes are atomic in MongoDB, which is why documents often embed related data (an order with its line items). If you must change several documents together, use a transaction (this needs a replica set):

JavaScript
// read-only: multi-document transaction
await client.withSession((session) =>
  session.withTransaction(async () => {
    await accounts.updateOne({ _id: 'a' }, { $inc: { balance: -50 } }, { session });
    await accounts.updateOne({ _id: 'b' }, { $inc: { balance: 50 } }, { session });
  }),
);

Design the model so that most operations touch one document; reach for transactions only when you truly need them.

What a document store does: a runnable stand-in

To see what is behind filters and update operators, here is a small collection class. It supports equality, comparison operators, $in, $or, dotted paths, and the $set, $inc and $push update operators. It is not a database (no disk, no indexes), but the semantics of the matching logic are the same idea.

Node.js A tiny Mongo-style collection

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Notice that toArray() returns a clone. The real driver also hands you copies from the network, so changing a result never changes the stored data until you call an update.

Redis: fast shared state

Redis keeps data in memory and answers in well under a millisecond. It is not just a string cache: values can be strings, hashes, lists, sets, sorted sets, streams and more. Commands are tiny and atomic, which is why Redis is used for caches, sessions, counters, rate limits, locks and job queues.

The redis package (node-redis) is the official client. Read-only:

JavaScript
// read-only: requires "npm install redis" and a running Redis server
import { createClient } from 'redis';

const redis = createClient({ url: process.env.REDIS_URL });
redis.on('error', (err) => console.error('redis error:', err.message)); // never leave this out
await redis.connect();

await redis.set('user:42', JSON.stringify({ name: 'Ada' }), { EX: 300 });  // expires in 5 minutes
const cached = await redis.get('user:42');            // string, or null on a miss
await redis.del('user:42');

await redis.incr('visits:home');                      // atomic counter
await redis.hSet('session:abc', { userId: '42', role: 'admin' });
const role = await redis.hGet('session:abc', 'role');

// a simple lock: only one process wins SET ... NX
const gotLock = await redis.set('lock:report', 'worker-1', { NX: true, EX: 30 });

await redis.quit();   // finishes pending commands, then closes (newer versions also offer close())

Details that matter in production:

  • Values are strings (or bytes). You serialize objects with JSON.stringify and parse them back. A hash (hSet) stores flat fields and lets you read one field without parsing the rest.
  • Always set a TTL on cache keys (EX seconds, PX milliseconds). A cache without expiry is a memory leak with a nicer name.
  • The error event handler is mandatory. The client emits error when the connection drops. An unhandled error event crashes the process (see Events and EventEmitter). The client reconnects by itself; your handler is for logging.
  • Design keys on purpose. Use a namespace and colons: app:user:42, app:cart:42. Include a version when the cached shape changes (v2:user:42) so old and new code do not read each other's data.
  • Do not use KEYS * in production. It scans every key and blocks the server. Use SCAN to iterate in small steps.
  • Keep values small. A multi-megabyte value takes time to transfer and blocks other commands while it is processed.

Caching with Redis: the cache-aside pattern

In Caching in Memory you cached inside one process. That works until you run two instances: each has its own cache, so they disagree and each hits the database. A Redis cache is shared by every instance and survives deploys of the app.

The standard recipe is cache-aside:

  1. Read from the cache. On a hit, return it.
  2. On a miss, load from the database.
  3. Store the result in the cache with a TTL, then return it.
  4. When the data changes, delete the cache key (invalidate), so the next read reloads fresh data.

One more rule: protect against a stampede. If a popular key expires and 500 requests arrive at once, all 500 miss and all 500 query the database. Within one process, you can share a single in-flight promise per key, as you saw in the caching lesson. With many instances you can use a short Redis lock (SET key value NX EX) so that only one instance reloads while others wait briefly or serve slightly stale data.

The stand-in below uses a fake Redis client with a manual clock, so you can watch TTL expiry without waiting. The cache-aside function is exactly what you would write against the real client, because the fake has the same get, set (with EX) and del calls.

Node.js Cache-aside with TTL and stampede control

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

There is a subtle race in plain cache-aside: a slow reader can load the old row, a writer then updates the database and deletes the key, and finally the slow reader stores the old value in the cache. The TTL limits how long that stale value lives, which is one more reason never to cache without an expiry. For data where staleness is unacceptable, do not cache it, or use short TTLs.

Connection handling

Both drivers manage their own connections. The most common mistake is to create a client per request.

  • Create one client per process at startup and share it (export it from a module; see Module Patterns). MongoClient keeps a pool of connections internally and reuses them. The Redis client multiplexes commands over one connection by default, which is enough for most apps; use extra connections for blocking commands or Pub/Sub, which need their own.
  • Connect before you accept traffic. Await connect() during startup. If it fails, exit with a clear message instead of serving errors.
  • Set timeouts. Server selection (MongoDB), connect and command timeouts (Redis) keep a dead database from freezing every request.
  • Plan for the cache being down. A cache is an optimization. If Redis fails, catch the error, log it, and fall back to the database. Never let a failing cache take the site down.
  • Close cleanly on shutdown. On SIGTERM stop accepting requests, wait for the running ones, then call client.close() and redis.quit(). See Graceful Shutdown and Health Checks.
  • Keep secrets out of code. Connection strings contain passwords; read them from the environment (Configuration, Secrets and Environment).

Choosing between them

Use MongoDB when your data is naturally document-shaped, varies between records, and is mostly read and written as a whole unit. Use Redis for data that is small, hot, temporary or needs atomic counters and expiry. Use neither as a reason to skip thinking about the data model: if you find yourself joining collections by hand in application code, a relational database (SQLite is a good place to learn) may have been the better choice. For ORMs on top of relational databases, see ORMs: Prisma and Sequelize.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In MongoDB, what does the filter { tags: 'vip' } match when tags is an array field?

    Matching a scalar against an array field means 'the array contains this value'. Exact-array matching needs the full array as the filter value.
  2. 2. How many MongoClient instances should a typical Node.js server create?

    A MongoClient owns a connection pool. Creating one per request throws away connection reuse and can exhaust the server's connection limit.
  3. 3. Why should cache keys almost always be written with an expiry (TTL)?

    Memory is limited and data changes. A TTL bounds both the memory used and how long a stale value can survive a missed invalidation.
  4. 4. What happens if a Redis client emits an 'error' event and nobody listens for it?

    EventEmitter treats an unhandled 'error' event as fatal. Always attach an error listener to the client, usually just to log.
  5. 5. In the cache-aside pattern, what should you do after updating a row in the database?

    Deleting the key is the simplest safe invalidation. The next read misses, loads the new row and re-caches it.

Summary

  • MongoDB stores documents in collections; filters and updates are objects with $ operators, cursors are lazy, and indexes matter as much as in SQL.
  • Redis is an in-memory store with atomic commands; values are strings (serialize JSON yourself), keys deserve a namespace and a version, and cache keys need a TTL.
  • Create one client per process, connect at startup, set timeouts, attach an error handler, and close both clients on shutdown.
  • Cache-aside means read cache, fall back to the database, store with a TTL, and delete the key when data changes. Share in-flight work to prevent a stampede.
  • Treat a cache as optional: if it fails, fall back to the database.

Next: ORMs: Prisma and Sequelize