Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 37 ยท Aggregation

The Aggregation Pipeline Basics

Learn Model.aggregate() in Mongoose - how pipeline stages transform documents, the core stages $match, $project, $sort, $limit and $count, and a runnable pipeline stand-in.

find() returns documents as they are stored. Often you want something else: total sales per month, the average rating per product, the ten most active users. That is aggregation: computing new, summarized or reshaped results inside the database.

MongoDB does this with an aggregation pipeline. You send an array of stages; documents flow through the stages one after another, and each stage transforms the stream. Mongoose exposes it as Model.aggregate([...]).

Pipeline thinking

Imagine an assembly line. The first stage receives the whole collection. It passes some documents (or new documents) to the second stage, and so on. The output of the last stage is the result.

flowchart LR
  C["orders collection"] --> M["$match<br/>status = paid"] --> G["$group<br/>by customer"] --> S["$sort<br/>total desc"] --> L["$limit 3"] --> R["result"]

Each stage reads the previous stage's output.

JavaScript
const top = await Order.aggregate([
  { $match: { status: 'paid' } },
  { $group: { _id: '$customer', total: { $sum: '$amount' } } },
  { $sort: { total: -1 } },
  { $limit: 3 },
]);

Two points make aggregation different from find. First, the result is plain objects, not Mongoose documents: no virtuals, no getters, no casting through your schema's methods. Second, Mongoose does not cast values in a pipeline the way it does in find: a string id in $match will not match an ObjectId field. Convert yourself with new mongoose.Types.ObjectId(id).

The core stages

Stage Purpose
$match keep documents that satisfy a filter (same syntax as find)
$project choose, rename or compute fields
$sort order documents
$limit / $skip page through results
$count replace the stream with one document holding a count
$addFields / $set add computed fields, keep the rest
$unwind turn an array field into one document per element
$group combine documents by a key and compute totals
$lookup join another collection

This lesson covers the first half; group and lookup covers the rest.

A tiny pipeline engine

You cannot run MongoDB here, so the stand-in implements $match, $sort, $limit and $project over an array. The point is the shape: every stage is a function from an array of documents to an array of documents, and the pipeline is a fold over the stages.

Node.js Stages as functions (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

$match first, always

MongoDB can use an index only when $match (and $sort) are at the start of the pipeline. Put $match as early as possible so later stages process fewer documents. The optimizer reorders some stages for you, but do not rely on it: write the filter first.

Node.js Order matters for work done (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

$project and computed fields

$project keeps listed fields (value 1), drops _id with _id: 0, and can compute values with expressions using $ to read a field and operators like $multiply, $concat, $toUpper, $cond.

JavaScript
await Product.aggregate([
  { $match: { active: true } },
  {
    $project: {
      _id: 0,
      name: 1,
      priceWithTax: { $multiply: ['$price', 1.2] },
      label: { $concat: ['$name', ' (', '$sku', ')'] },
    },
  },
]);

Use $addFields (alias $set) instead when you want to keep all existing fields and only add new ones. $unset removes fields. Prefer these three over one giant $project.

$count and counting patterns

$count replaces the stream with one document: { $count: 'total' } gives [{ total: 17 }]. When you only need a number and a simple filter, Model.countDocuments(filter) is shorter and exactly as fast. Use $count when counting at the end of a longer pipeline.

Running and inspecting pipelines in Mongoose

Model.aggregate() returns an Aggregate object. It is thenable, so await runs it. You can chain helper methods instead of writing the array:

JavaScript
const result = await Order.aggregate()
  .match({ status: 'paid' })
  .group({ _id: '$customer', total: { $sum: '$amount' } })
  .sort('-total')
  .limit(3);

To see how MongoDB runs a pipeline, call .explain('executionStats') and read whether an index was used (covered in Indexes in Mongoose). Stages have a 100 MB memory limit unless you pass { allowDiskUse: true }.

Common mistakes

  • Forgetting to cast ids ('$match': { _id: idString } returns nothing).
  • Using a Mongoose field name that differs from the stored name (aliases and select: false do not apply).
  • Putting $sort after $group on a large collection where it could have used an index earlier.
  • Expecting virtuals, getters or toJSON transforms on the output.
  • Returning huge arrays: add $limit and a $project to shrink the data sent over the wire.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does Model.aggregate() return for each result?

    Aggregation results are plain objects; virtuals, getters and instance methods are not applied.
  2. 2. Why should $match come first in a pipeline?

    An early $match can use indexes and shrinks the stream that every later stage must process.
  3. 3. A $match on _id with a string finds nothing. Why?

    Convert with new mongoose.Types.ObjectId(id); pipeline stages are not cast through the schema.
  4. 4. Which stage adds a computed field while keeping all existing fields?

    $addFields (alias $set) keeps every existing field and adds the new ones.

Summary

  • A pipeline is an array of stages; each transforms the stream from the one before.
  • Core stages: $match, $project, $addFields, $sort, $limit, $skip, $count.
  • Put $match and $sort first so indexes work and work is reduced.
  • Results are plain objects and ids are not cast automatically.
  • Chainable helpers (.match().group().sort()) build the same array.

Next: Aggregation: group and lookup