Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 70 ยท Aggregation pipeline

Map-Reduce and Why Pipelines Replaced It

Learn how MongoDB's legacy mapReduce works (map, reduce, finalize, out), why it is deprecated, and how to rewrite each job with $group, $merge, $accumulator and $function.

Before the aggregation pipeline existed, MongoDB had one tool for computing totals and summaries over a collection: map-reduce. You wrote two JavaScript functions, map and reduce, and the server ran them over your data. You will still meet mapReduce in older code, in books and in interview questions, so it is worth understanding. But it has been deprecated since MongoDB 5.0, and almost everything it does is now done faster, and in a more readable way, with an aggregation pipeline.

This lesson explains how map-reduce works, shows a complete example, lists its limitations, and then rewrites the same job with $group, $merge and, for the rare custom case, $accumulator. The mapReduce and mongosh snippets are real MongoDB syntax and cannot run on this page. The TryIts use a stand-in written in plain JavaScript that follows the map-reduce model, so you can run the idea and compare it with a pipeline. It is not a MongoDB server.

db.collection.mapReduce() is deprecated. Do not start new projects with it. Use aggregation pipelines instead, which are faster, support indexes and are easier to read, test and optimize.

The idea: map, group, reduce

Map-reduce splits a calculation into steps, borrowed from distributed computing:

  1. Map: the map function runs once per input document. It calls emit(key, value) zero or more times, producing key-value pairs. For "total sales per customer" you would emit (customerId, amount) for each order.
  2. Group by key: the server collects all values emitted for the same key into an array.
  3. Reduce: the reduce(key, values) function turns each array of values into one value. For sales, it adds the amounts.
  4. Finalize (optional): a finalize(key, reducedValue) function post-processes each final value, for example to compute an average from a sum and a count.
  5. Output: the results are returned inline or written to a collection.
JavaScript
db.orders.mapReduce(
  function () { emit(this.customerId, this.amount); },          // map
  function (key, values) { return Array.sum(values); },         // reduce
  {
    query: { status: "paid" },                                  // only paid orders go to map
    out: { inline: 1 }                                          // return results (or { merge: "totals" })
  }
)

Inside map, this is the current document, and emit is provided by the server. The result is an array of { _id: key, value: reducedValue } documents.

Rules your reduce function must follow

Because MongoDB may call reduce several times for the same key (for example once per batch of values, and then again on the partial results), the function must be:

  • Associative and commutative: reducing in any grouping or order gives the same answer.
  • Idempotent: reduce(key, [reduce(key, valuesA), ...valuesB]) must equal reduce(key, [...valuesA, ...valuesB]).
  • Type-stable: it must return the same shape as the values that map emits. If you emit { sum: 5, count: 1 }, reduce must return { sum, count }, not a bare number.
  • Not called for single values. If a key has only one emitted value, reduce is skipped entirely, and that value goes straight to finalize.

An average is the classic trap. reduce returning sum / count breaks the rules, because averaging averages gives the wrong answer. The correct pattern is to emit { sum: amount, count: 1 }, add both fields in reduce, and divide in finalize.

Try it: map-reduce on a few orders

The stand-in below implements mapReduce the way the server describes it, including calling reduce in batches of two values so you can see that a correct reduce function survives re-reduction. It runs two jobs: total per customer, and average order amount per customer using finalize.

JS map, reduce and finalize

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Customer cy has a single order, so its reduce never ran; and the { sum, count } shape was the same for every value, which is why finalize could treat single and combined values the same way.

Why it was replaced

mapReduce still works in older server versions, but the pipeline is better in nearly every way:

mapReduce Aggregation pipeline
Where it runs JavaScript engine in the server Native C++ stages
Speed Slow (document conversion between BSON and JavaScript, single-threaded JS) Much faster, optimizer reorders and pushes down work
Indexes Only for the initial query filter Used by $match, $sort and more, at any point where it helps
Readability Two functions with strict rules A list of named stages
Safety Server-side JavaScript must be enabled (a security risk, see NoSQL injection) No JavaScript by default
Output Inline (16 MB limit) or a collection Cursor, or $out/$merge into a collection
Status Deprecated since 5.0 The supported way

Rewriting map-reduce as a pipeline

Most map-reduce jobs are a $group. The map function's emit(key, value) becomes the group's _id, the reduce function becomes an accumulator, and the query becomes $match:

JavaScript
db.orders.aggregate([
  { $match: { status: "paid" } },                         // query
  { $group: { _id: "$customerId", total: { $sum: "$amount" } } }   // map + reduce
])

The average with finalize is even shorter, because $avg is built in:

JavaScript
db.orders.aggregate([
  { $match: { status: "paid" } },
  { $group: { _id: "$customerId", average: { $avg: "$amount" }, orders: { $sum: 1 } } }
])

The common translations are:

Map-reduce piece Pipeline equivalent
query option $match first
sort and limit options $sort, $limit
emit(key, value) the _id of $group, with the value inside an accumulator
reduce adding numbers $sum; keeping extremes: $min, $max; building lists: $push, $addToSet
finalize computing a ratio $avg, or a $project/$set stage after $group
emit several times per document $unwind an array first, then $group
out: { merge: "totals" } or { reduce: ... } $merge as the last stage
out: { replace: "totals" } $out as the last stage

Writing the result into a collection, the way out did, uses $merge:

JavaScript
db.orders.aggregate([
  { $match: { status: "paid" } },
  { $group: { _id: "$customerId", total: { $sum: "$amount" }, orders: { $sum: 1 } } },
  { $merge: { into: "customer_totals", on: "_id", whenMatched: "replace", whenNotMatched: "insert" } }
])

$merge can update an existing output collection incrementally, which was map-reduce's out: { reduce: ... } use case (an incremental job that processes only new orders each night).

When you need custom JavaScript

If the calculation truly needs custom logic that no accumulator covers, MongoDB 4.4 added $accumulator and $function, which run your JavaScript inside a $group or any expression. They need server-side JavaScript enabled and are slower than built-in operators, so use them only as a last resort. See the aggregation expressions first, because built-ins cover almost every case.

The second TryIt runs the equivalent pipeline stages (a $match and a $group) over the same orders and compares the output with the map-reduce result from the first example.

JS The same results with $match and $group

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The numbers match the earlier output: ann 70 (average 23.33), bob 50 (25), cy 90 (90). Besides being shorter, the pipeline can start with a $match that uses an index and can be inspected with explain.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does the map function of mapReduce do?

    map runs per document and emits key-value pairs. reduce later combines the values for each key.
  2. 2. Why must a reduce function return the same shape as the values passed to emit?

    Reduce can be applied to its own output in later passes, so its result must be usable as an input value.
  3. 3. Which pipeline stage replaces most map-reduce jobs?

    $group uses _id as the emit key and accumulators such as $sum, $avg and $push in place of reduce.
  4. 4. How do you write pipeline results to a collection, like mapReduce's out option?

    $out replaces a collection; $merge can insert, replace or merge into an existing one.
  5. 5. What is the status of mapReduce today?

    mapReduce is deprecated; use aggregation pipelines, and $accumulator or $function if custom JavaScript is unavoidable.

Summary

  • Map-reduce splits a job into map (emit key-value pairs), reduce (combine values per key) and an optional finalize, with output inline or in a collection.
  • Reduce may be called repeatedly on partial results, so it must be associative, idempotent and return the same shape it receives; averages need a { sum, count } shape.
  • mapReduce is deprecated since MongoDB 5.0: it is slower, harder to read and needs server-side JavaScript.
  • Rewrite jobs with $match + $group (and $unwind for multiple emits), and use $out or $merge to store results.
  • $accumulator and $function cover rare custom logic, but built-in operators should come first.

Next, put the pipeline to work on a bigger scale in Aggregation Performance.