Skip to content
Tutorials Center
My learning

Topics

Lesson 51 of 70 ยท Replication, sharding and operations

Sharding

Learn MongoDB sharding: shards, mongos and config servers, choosing a shard key, ranged vs hashed sharding, chunks, the balancer, zones and resharding.

A replica set copies the same data onto several servers. That protects you from failure, but it does not let you store more than one server can hold or handle more writes than one primary can take. When one machine is no longer enough, MongoDB can shard: split a collection into pieces and spread the pieces over several groups of servers, each called a shard. To your application the collection still looks like one.

Sharding is powerful and permanent in its consequences. A good choice of shard key gives you nearly linear growth, and a bad one gives you a hot server, slow queries and a painful migration. This lesson explains the parts of a sharded cluster, how the shard key decides where each document lives, how to judge a key, and what the balancer, zones and resharding do. It builds on Replica Sets and on How Indexes Work.

A sharded cluster needs many servers, so none of it can run on this site. The mongosh listings are real commands and are read-only. The runnable examples are plain JavaScript stand-ins: one spreads sample orders over three imaginary shards using different shard keys, and the other decides which shards a query would have to visit. The hash function and the chunk boundaries are simplified.

The parts of a cluster

Shards Each holds a slice of the data, as a replica set.
mongos Routers that your application talks to.
Config servers A replica set that stores the chunk map.
1 / 2

The parts of a sharded cluster

A sharded cluster has three kinds of component:

  • Shards. Each shard stores part of the data. In production every shard is itself a replica set, so each slice is highly available.
  • mongos routers. Applications connect to mongos, not to the shards. A router looks up which shard holds the data a query needs, forwards the request, and merges the answers. You run several routers; they are stateless.
  • Config servers. A replica set that stores the cluster's metadata: which ranges of the shard key live on which shard.

Your driver connects to a mongos with a normal connection string, and the code does not change. You turn on sharding for a collection with sh.shardCollection():

JavaScript
// MongoDB 6.0 and later: sharding a collection is enough.
// (Older versions also needed sh.enableSharding("shop") first.)

// Ranged sharding on a single field
sh.shardCollection("shop.orders", { customerId: 1 })

// Hashed sharding
sh.shardCollection("shop.orders", { customerId: "hashed" })

// A compound shard key
sh.shardCollection("shop.orders", { country: 1, customerId: 1 })

A collection that is not sharded lives entirely on one shard, the database's primary shard, so sharding is chosen collection by collection. The shard key needs a supporting index. If the collection already has data, you must create that index first.

The shard key decides everything

The shard key is one or more fields present in every document. MongoDB divides the range of shard key values into contiguous chunks and assigns chunks to shards. When you insert a document, the router reads its shard key and sends it to the shard that owns that value. The key is the single most important design decision, and three properties describe a good one.

Cardinality is the number of distinct values. If you shard on country and there are only 200 countries, there can never be more than 200 chunks, and all documents for one country must sit on one shard. A low-cardinality key puts a hard ceiling on how far you can scale.

Frequency is how often each value appears. If 55% of orders come from one country, the shard that owns that country holds more than half the data and takes more than half the traffic. A chunk whose documents all share one key value cannot be split at all. MongoDB calls such a chunk jumbo, and it cannot be moved by the balancer in the normal way.

Rate of change is whether new values are monotonically increasing, like an ObjectId, a timestamp or an order number. Each new document has a larger key than every earlier one, so each insert lands in the last chunk, which belongs to a single shard. That shard receives all inserts while the others sit idle, a hot shard.

A good shard key also appears in most of your queries. If a query contains the shard key, the router can send it to one shard (a targeted query). If it does not, the router must ask every shard and merge the results, which is a scatter-gather query. Scatter-gather works, but its cost grows with the number of shards.

Ranged and hashed sharding

With ranged sharding the key values are kept in order, and a chunk covers a contiguous range such as customerId 100 to 199. Documents with close key values are stored together, so a range query like "customers 150 to 250" touches only a few shards. The risk is that skewed or monotonic keys cluster on one shard.

With hashed sharding MongoDB stores the hash of the key and shards on that. Hashing scrambles the values, so even a monotonic key like an ObjectId spreads over all shards evenly. The price is that neighbors in key order are no longer neighbors: an equality query is still targeted, but a range query on the key becomes scatter-gather. Before MongoDB 7.2 a hashed key had to be a single field; newer versions also allow a hashed field inside a compound key.

You can also build a compound key, such as { country: 1, customerId: 1 }. The first field can have low cardinality, because the second field adds the missing granularity. Queries that include country are targeted, and each country can still be split across many chunks.

Choosing between ranged and hashed

Ranged Hashed
Good at Range queries on the key (customerId from 150 to 250) hit only a few shards; related documents sit together (data locality) Writes spread evenly, even for ever-increasing keys such as ObjectIds or timestamps; fewer hotspots
Weak at A monotonic key (a timestamp, an ObjectId) sends every new insert to the last chunk, one hot shard Range queries on the key become scatter-gather
Gives you Fine-grained control: you can split ranges and place them on shards with zones (below) An automatic, balanced layout with little tuning
Pick it when Queries are range-based or you need location control Writes dominate and the key is high-cardinality and mostly looked up by equality

How the router handles a query follows from this: mongos reads the chunk map cached from the config servers, sends an equality or range query on the shard key only to the shards that own those values, and merges the replies. It also routes writes, so applications never need to know which shard holds a document. If the shard key is missing from the query, the router asks every shard.

The first TryIt: comparing shard keys

The stand-in below creates 900 sample orders. The first 300 exist when sharding begins, and their key values set the chunk boundaries (three even slices of those values, one per shard). The next 600 orders are inserted afterwards, and the program counts where they go. Look at the bars, then change the data: for example make customerId sequential, or give country more values.

JS Where do 600 new orders go?

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The monotonic orderId sends all 600 new inserts to the last shard, even though the same key looked perfectly balanced when the cluster was created. The random customerId spreads evenly in ranged mode and also when hashed. country gives lopsided shards because of its few values and its skew, and the compound key country + customerId fixes that by adding granularity within each country. Each strategy answers a different question, so ask three of them for every candidate key: how many distinct values, how uneven, and how fast it grows.

Chunks and the balancer

A chunk is a contiguous range of shard key values owned by one shard. When a chunk grows past a size limit (128 MB by default in current versions), it is split into two. The balancer, a background process on the config server primary, watches how data is distributed and migrates chunks from the fullest shard to the emptiest, so that the cluster stays even when you add a shard or the data grows unevenly. Since MongoDB 6.0 it balances by data size instead of chunk count.

Migrations copy data while the cluster stays online, but they do cost I/O. You can see the state with sh.status() and control the balancer with sh.stopBalancer(), sh.startBalancer() and balancing windows that restrict migrations to quiet hours. Adding capacity is simple: add a shard with sh.addShard(), and the balancer fills it over time.

Zones: placing data on purpose

Zone sharding lets you pin ranges of the shard key to specific shards. You tag shards with a zone name and tie shard key ranges to that zone, and the balancer keeps the data where you asked. Typical uses are data residency (EU customers on EU servers) and tiering (recent data on fast hardware):

JavaScript
sh.addShardToZone("shardEU1", "EU")
sh.addShardToZone("shardUS1", "US")

sh.updateZoneKeyRange(
  "shop.orders",
  { country: "DE", customerId: MinKey() },
  { country: "DE", customerId: MaxKey() },
  "EU"
)

This works well with a compound shard key that starts with the field you place by (country here).

Changing your mind: refining and resharding

Choosing the key is hard, because early data rarely shows later behavior. MongoDB has two escape hatches. refineCollectionShardKey (4.4 and later) adds one or more suffix fields to the existing key. This gives a coarse key more granularity without moving data, for example turning { country: 1 } into { country: 1, customerId: 1 }. reshardCollection (5.0 and later) changes the key entirely and rewrites the collection online:

JavaScript
db.adminCommand({
  reshardCollection: "shop.orders",
  key: { customerId: "hashed" }
})

Resharding is a heavy operation that needs spare disk space and I/O. Treat it as a rescue tool, not a plan. It is far better to spend time on the key up front, using real query logs and a sample of production data.

The second TryIt: targeted or scatter-gather?

The router decides which shards a query must visit. The stand-in below models a chunk table for a ranged key and for a hashed key on customerId. It routes three queries and tells you whether each is targeted. Try your own filters, or change the chunk boundaries.

JS Which shards does this query visit?

Press Run (or Ctrl + Enter) to see the console output here.

No output.

With the ranged key, an equality query goes to exactly one shard and a range query visits only the shards whose chunks overlap it. With the hashed key the equality query is still targeted, but the range query must ask everyone. A filter that leaves out the shard key, such as status, is scatter-gather under both. If your important queries all filter on fields other than the shard key, the key is wrong for your workload.

When to shard, and when not to

Shard when your data or write load truly does not fit one replica set: the working set no longer fits in RAM on the biggest affordable machine, or one primary cannot absorb the write rate. Before that, try cheaper steps. Add the right indexes (Designing Compound Indexes), fix schema problems from Common Performance Pitfalls, archive old data, and scale the machine up. A sharded cluster multiplies operational complexity, and many applications never need one.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the job of a mongos router?

    Applications connect to mongos. It consults the chunk metadata kept on the config servers and forwards the request. Moving chunks is the balancer's work.
  2. 2. You shard an orders collection on the ObjectId _id with ranged sharding. What problem do you expect?

    ObjectIds increase over time, so new documents always fall in the highest chunk and one shard takes all the writes. A hashed key spreads them out.
  3. 3. Which shard key has a cardinality problem?

    A key with few distinct values cannot be divided into many chunks, which caps scaling and makes uneven shards likely. A compound key adds the missing granularity.
  4. 4. A query filters only on a field that is not part of the shard key. What happens?

    Without the shard key (or a prefix of it) the router cannot know which chunks matter, so it queries all shards. It works but costs more as shards are added.

Summary

  • Sharding splits a collection across several shards (each normally a replica set); mongos routers send requests and config servers keep the chunk map.
  • The shard key determines where each document lives; judge it by cardinality, frequency and change rate, and by whether common queries include it.
  • Ranged keys keep neighbors together and suit range queries but can hot-spot; hashed keys spread load evenly but make key range queries scatter-gather.
  • Compound keys add granularity to a low-cardinality first field.
  • Chunks are key ranges; the balancer moves chunks to even out shards, and zones pin ranges to chosen shards.
  • refineCollectionShardKey extends a key and reshardCollection changes it online, but both are heavy; shard only when one replica set truly cannot cope.

Next, learn how to protect your data in Backup, Restore and Import/Export.