When You Actually Need Sharding

Sharding is MongoDB’s mechanism for horizontal scaling: instead of storing your entire dataset on one replica set, you split it across multiple replica sets called shards, with a routing layer (mongos) sending each operation to whichever shard actually holds the relevant data. It sounds like the natural next step once an application gets busy, but sharding is also the most operationally expensive decision you can make in a MongoDB deployment — it adds new processes to run and monitor, and it forces you to pick a shard key that is extremely difficult to change later. This lesson is less about how to flip the sharding switch and much more about how to tell whether you actually need to.

Overview: How Sharding Fits Into MongoDB’s Scaling Story

A normal MongoDB deployment is a replica set: one primary node that accepts all writes, plus one or more secondaries that replicate the primary’s data. Every member of a replica set holds a full copy of every document. Replica sets solve high availability (a secondary can be promoted if the primary fails) and can offload read traffic to secondaries, but they do not solve a capacity problem — every node still has to store, and often hold in RAM, the entire dataset.

A sharded cluster is different. Data is partitioned across multiple shards — and each shard is itself a full replica set for redundancy — so no single machine needs to hold the whole collection. Three components make this work: the shards themselves, a set of config servers that store the metadata describing which ranges of data (called chunks) live on which shard, and one or more mongos router processes that your application actually connects to. The application never talks to a shard directly; mongos consults the config servers’ chunk map and forwards each query to the correct shard, or shards.

The single most consequential decision in sharding is the shard key — the field, or fields, MongoDB uses to decide which chunk, and therefore which shard, a document belongs to. Every document in a sharded collection is routed by its shard key value. Choose well, and writes and reads spread evenly across shards. Choose poorly, and you can end up with all the traffic hitting one shard anyway — all the operational overhead of sharding with none of the benefit. Since MongoDB 5.0, reshardCollection lets you change a shard key without a fully manual migration, but it is still a slow, resource-intensive background operation on a large collection, not a safety net for a rushed initial choice.

The Real Signals That You Need Sharding

Before reaching for sharding, rule out cheaper options: adding a covering index, moving read traffic to secondaries, archiving old data out of the hot collection, or simply scaling the existing replica set vertically with more RAM, faster NVMe disks, or more CPU. Vertical scaling and better indexing solve the majority of “MongoDB feels slow” problems. Sharding is the answer to a narrower, harder problem: a single replica set’s primary no longer has enough capacity, in one of three specific ways.

  • Working set exceeds RAM. MongoDB’s performance depends heavily on keeping the “working set” — the indexes and actively-accessed documents — in memory in the WiredTiger cache. When your working set is bigger than available RAM on the largest instance you’re willing to run, the server starts hitting disk constantly and latency climbs no matter how good your indexes are.
  • Dataset exceeds practical disk capacity. Even with cheap disk, a single replica set has a practical ceiling: backup time, initial sync time for a new secondary, and restore time during an incident all grow with data size. Multi-terabyte collections that keep growing are a classic sharding trigger.
  • Write throughput exceeds one primary’s capacity. In a replica set, all writes go through a single primary. If you’ve already optimized your write path (bulk writes, an appropriate write concern, no unnecessary indexes) and you’re still saturating that one primary’s disk I/O or CPU, sharding lets you accept writes on multiple shards in parallel.

If you can’t point to one of these three and back it up with actual numbers from db.stats(), db.serverStatus(), or your monitoring dashboards, you almost certainly don’t need sharding yet.

Syntax

Two commands turn on sharding for a collection. Both are run in mongosh connected to a mongos router, on a deployment where sharding has already been enabled at the cluster level (config servers and at least one shard already added).

sh.enableSharding("<database>");
sh.shardCollection("<database>.<collection>", { <shardKeyField>: 1 });
  • sh.enableSharding("<database>") — allows collections in that database to be sharded. A one-time, per-database setup step.
  • sh.shardCollection(ns, key) — ns is the full "database.collection" namespace; key is the shard key document, in the same shape as an index specification.
  • A shard key value of 1 or -1 creates a ranged shard key — documents are grouped into contiguous chunks by value order. Good when you also query by ranges on that field.
  • A shard key value of "hashed" creates a hashed shard key — MongoDB hashes the field’s value before assigning a chunk, spreading writes evenly even if the underlying field is monotonically increasing. Trades away efficient range queries on that field.
  • The shard key must be backed by an index (MongoDB creates one automatically if none exists), and its fields cannot be changed later without a full reshardCollection operation.
Shard key type Write distribution Range queries Best for
Ranged ({ field: 1 }) Even only if field values are naturally scattered Efficient — targets a contiguous set of shards High-cardinality fields like customerId or region that queries also filter or sort by
Hashed ({ field: "hashed" }) Even, regardless of input pattern Scatter-gather — every shard must be queried Monotonic fields, like timestamps or ObjectIds, where write distribution matters more than range scans

Examples

Example 1: Check before you shard

db.stats();
{
  db: 'ecommerce',
  collections: 42,
  views: 3,
  objects: 184320551,
  avgObjSize: 612,
  dataSize: 112812097332,
  storageSize: 48302981120,
  indexes: 96,
  indexSize: 21504331776,
  ok: 1
}

Here dataSize is roughly 112 GB and indexSize is about 21 GB — combined, well over 130 GB of working set. If the largest instance the team is willing to run tops out around 64 GB of RAM, that’s real evidence the working set no longer fits in memory: a genuine sharding signal, not a guess. Always look at this number before writing a single line of sharding configuration.

Example 2: Sharding a real, growing collection

sh.enableSharding("ecommerce");
sh.shardCollection("ecommerce.orders", { customerId: 1, orderDate: 1 });
{
  collectionsharded: 'ecommerce.orders',
  ok: 1
}

This shards the orders collection using a compound shard key: customerId first, then orderDate. Because most application queries filter by customerId (a single customer viewing their own order history), mongos can route those queries to a single shard instead of asking every shard. orderDate as the second key keeps each customer’s orders sorted usefully within their chunk range without hurting write distribution, since customerId values are scattered across many different customers.

Example 3: Targeted query vs. scatter-gather

db.orders.find({ customerId: "CUST-88213" }).explain("executionStats");
{
  queryPlanner: {
    winningPlan: {
      stage: 'SINGLE_SHARD',
      shards: [ { shardName: 'shard02', ... } ]
    }
  },
  executionStats: { nReturned: 47, totalDocsExamined: 47 }
}

Because the query includes an equality match on the full shard key prefix (customerId), mongos can compute exactly which shard owns that value and send the query there alone — a targeted query. Compare that to a query that omits the shard key entirely, such as db.orders.find({ status: "pending" }): with no shard key in the filter, mongos has no way to know which shard holds matching documents, so it fans the query out to every shard and merges the results, which costs far more than a targeted query even though the result set may be small.

How It Works Step by Step

When your application sends a query through mongos, here’s what actually happens:

  • mongos checks the query filter against the collection’s shard key ranges, using chunk metadata cached from the config servers.
  • If the filter includes an equality condition on the full shard key (or its prefix, for a compound key), mongos resolves it to one specific shard and forwards the query there directly — the targeted path shown in Example 3.
  • If the shard key isn’t present in the filter, or only a range is given, mongos must broadcast the query to every shard that could possibly hold matching data, then merge the returned results into a single sorted stream before handing them back to the driver.
  • In the background, a balancer process continuously watches chunk sizes across shards and migrates chunks from over-loaded shards to under-loaded ones, keeping data (and therefore load) roughly even. A poorly chosen shard key fights this process constantly rather than settling into a stable distribution.

Common Mistakes

Mistake 1: Sharding before you’ve hit a real ceiling

Turning on sharding for a 20 GB collection because the team is anticipating future growth adds config servers, extra shard replica sets, and cross-shard query complexity for zero present benefit — and it locks in a shard key decision before you have real query patterns to base it on. Check db.stats() and your monitoring first, as in Example 1, and only shard once vertical scaling and indexing genuinely can’t keep up.

Mistake 2: A monotonically increasing shard key

sh.shardCollection("ecommerce.orders", { _id: 1 });

The default _id is an ObjectId, which embeds a creation timestamp and is therefore roughly monotonically increasing. With a ranged shard key on _id, every newly inserted document has a higher value than everything before it, so every insert lands in the same, highest chunk — on the same shard. All new writes hit one shard regardless of how many shards you have, and the balancer has to constantly split and migrate that one hot chunk. Use a hashed key instead so inserts distribute evenly:

sh.shardCollection("ecommerce.orders", { customerId: "hashed" });

Mistake 3: A low-cardinality shard key

sh.shardCollection("ecommerce.orders", { status: 1 });

If status only ever takes a handful of values ("pending", "shipped", "delivered"), MongoDB can create at most a few chunks, no matter how large the collection grows — each one can become a huge, unsplittable “jumbo chunk” pinned to a single shard. A shard key needs enough distinct values to be split into many small, movable chunks. Combine a low-cardinality field with a high-cardinality one instead:

sh.shardCollection("ecommerce.orders", { region: 1, customerId: 1 });

Best Practices

  • Prove a real capacity ceiling with db.stats(), db.serverStatus(), and monitoring data before sharding — don’t shard speculatively.
  • Pick a shard key with high cardinality, even distribution across values, and one that matches your most common query patterns so most reads become targeted queries.
  • Avoid a purely monotonically increasing shard key (raw timestamps, default ObjectId) unless you use a hashed shard key to counteract it.
  • Model your real query patterns before choosing a compound shard key — equality fields you’ll always filter by should come first in the key.
  • Test the shard key against a realistic data and traffic sample in staging before committing in production; changing it later via reshardCollection is expensive.
  • Keep every shard as its own properly configured replica set — sharding adds capacity, it doesn’t replace the durability replication provides.

Practice Exercises

  • Run db.stats() and db.<yourCollection>.stats() against a real or sample database. Based on dataSize and indexSize, would that collection’s working set fit in 16 GB of RAM? Justify your answer with the actual numbers.
  • Given a reviews collection where 90% of queries filter by productId, propose a shard key and explain whether it should be ranged or hashed, and why.
  • A team shards a events collection on { createdAt: 1 } and later notices one shard consistently has far more chunks and disk usage than the others. Explain why this happened and propose a fix.

Summary

  • Sharding partitions data across multiple shards (each a full replica set), coordinated by config servers and routed through mongos; it solves capacity problems, not availability ones.
  • Shard only when you have concrete evidence of one of three ceilings: working set exceeding RAM, dataset exceeding practical disk limits, or write throughput exceeding a single primary’s capacity.
  • The shard key determines whether queries become fast, targeted operations or slow scatter-gather fan-outs across every shard.
  • Monotonically increasing and low-cardinality shard keys both create hot, unsplittable chunks and defeat the purpose of sharding.
  • Reach for vertical scaling, indexing, and read replicas first; sharding is a powerful but operationally heavy tool reserved for a genuine capacity ceiling.