Sharding Explained: Why One Database Is Never Enough (Eventually)

Walkthrough of how big systems split their data — and why almost every engineer needs to understand this at some point

Sharding Explained: Why One Database Is Never Enough (Eventually)

If you’ve ever worked with a database that started out fast and simple, and slowly turned into something that takes forever to query, back up, or even restart — you already understand the problem this article is about.

This article is a practical and effective explanation of sharding — what it is, when you actually need it, how to choose the right way to split your data, and what problems show up once you do. 

Leanring Objectives: What you’ll know by the end of this

  • What is Sharding?

  • Why “just get a bigger server” stops working at some point

  • How to pick a good shard key (and what a bad one looks like)

  • The three common ways to distribute data across shards

  • The real problems sharding introduces — hotspots, cross-shard queries, consistency

  • How to talk about sharding clearly, whether in a design doc or an interview

Let’s start from the beginning.

Why does sharding even exist?

Every database has limits. Storage limits, write throughput limits, read throughput limits. A single well-provisioned Postgres instance today can handle a genuinely large amount of data — something like 50–70 terabytes of storage and tens of thousands of writes per second, depending on configuration. That’s more than most applications will ever need.

But some applications do outgrow it. Maybe storage keeps climbing. Maybe write traffic keeps climbing. Either way, queries get slower, backups take longer, and at some point you hit a ceiling that adding more RAM or a faster disk won’t fix.

The first instinct is usually vertical scaling — get a bigger machine. And that’s a reasonable first step. It buys you time. But it doesn’t buy you forever. Eventually, no single machine, however large, can keep up.

That’s the point where sharding enters the conversation.

Quick question for you: have you ever seen a database get “upgraded” more than once just to buy a few more months? If yes, you’ve already lived the exact problem sharding solves.

So what actually is sharding?

In plain words:

Sharding means splitting your data across multiple databases, so no single database holds everything.

Each piece is called a shard. Each shard is its own independent database — its own CPU, its own memory, its own storage, its own connections. Together, all the shards add up to your full dataset.

Here’s the idea in a simple diagram:

                 ┌─────────────┐
                 │ Application │
                 └──────┬──────┘
                        │
              (which shard has this data?)
                        │
        ┌───────────────┼───────────────┐
        │               │               │
   ┌────▼────┐    ┌─────▼────┐    ┌─────▼────┐
   │ Shard 1 │    │ Shard 2  │    │ Shard 3  │
   │ (DB)    │    │ (DB)     │    │ (DB)     │
   └─────────┘    └──────────┘    └──────────┘

Instead of one giant database groaning under the weight of everything, you now have several smaller databases, each handling a slice of the load. Need more capacity? Add another shard.

Sounds great, right? It is — but it comes with new questions you didn’t have before:

  • How do we decide what goes on which shard?

  • How do we know which shard to look in when we need something?

  • What happens if one shard gets way more traffic than the others?

Let’s go through these one at a time.

Step 1: Choosing a shard key

The shard key is the field you use to decide which shard a piece of data belongs to. For example, if you shard by user_id, then all of a user's data lives together on one shard.

A good shard key has three properties:

1. High cardinality — lots of unique values, so data can spread out. A field like user_id works well because there are millions of unique users. A field like is_premium (true/false) does not — you only ever get two possible groups.

2. Even distribution — the values should naturally spread out roughly equally. If you shard by signup date and most users are recent, all your traffic piles onto whichever shard holds “recent” data. That’s not even at all.

3. Query alignment — the shard key should match how you actually query your data. If your most common query is “get all posts by this user,” sharding by user_id means that query only ever touches one shard. If you sharded by something unrelated, that same query would have to check every shard.

Here’s a simple way to picture a bad vs. good choice:

Bad shard key: is_premium (boolean)
→ Only 2 possible groups
→ Once each group is full, you're stuck
Good shard key: user_id
→ Millions of unique values
→ Naturally spreads out
→ Matches "get user's data" queries

A question worth sitting with: if you had to pick a shard key for your own project’s biggest table right now, what would you pick — and would it actually match your most common query?

Step 2: Deciding how to distribute the data

Once you’ve picked a shard key, you need a strategy for mapping its values to actual shards. There are three common approaches.

Range-based sharding

You split the shard key into ranges. For example:

function getShardByRange(userId) {
  if (userId <= 10_000_000) return "shard-1";
  if (userId <= 20_000_000) return "shard-2";
  return "shard-3";
}

Simple to understand. But it has a real weakness: if user IDs increase over time, all your newest (and usually most active) users end up on the last shard, while earlier shards sit relatively idle. One shard ends up doing most of the work.

Hash-based sharding

Instead of ranges, you hash the shard key and use the result to pick a shard. This is the approach most production systems actually use.

function simpleHash(str) {
  let hash = 0;
  for (let i = 0; i < str.length; i++) {
    hash = (hash * 31 + str.charCodeAt(i)) >>> 0;
  }
  return hash;
}

function getShardByHash(userId, numShards) {
  const hash = simpleHash(String(userId));
  return `shard-${hash % numShards}`;
}

Hashing scrambles the input, so data spreads evenly across shards, regardless of whether the underlying IDs are sequential or clustered.

There’s one catch: if you ever add or remove a shard, the number you’re modding by (numShards) changes, and almost every key now maps to a different shard than before. That means a huge, disruptive reshuffle of data.

This is exactly the problem consistent hashing solves. Instead of a plain modulo, keys and shards are placed on a conceptual ring, and only a small portion of data needs to move when a shard is added or removed. It’s a deeper topic on its own, but the short version is: hash-based sharding + consistent hashing is the industry default, and for good reason.

Directory-based sharding

Here, you keep a lookup table that explicitly records which shard each key lives on.

const shardDirectory = {
  "user-101": "shard-1",
  "user-102": "shard-3",
  "user-103": "shard-2",
};

function getShardFromDirectory(userId) {
  return shardDirectory[userId];
}

This gives you a lot of flexibility — you can move any specific record to any shard whenever you want. But it adds an extra lookup on every single request, and that lookup table itself becomes a single point of failure. If it goes down, you don’t know where anything lives.

In short: range-based is simple but risky, hash-based is the reliable default, and directory-based is powerful but adds complexity you should only take on if you genuinely need that flexibility.

The problems sharding introduces

Sharding solves the “one database can’t handle it” problem. But it creates a few new ones. This is the part that’s easy to forget about until you actually run into it.

1. Hotspots

Even with a good shard key, some shards can still end up busier than others. Imagine one particular user becomes extremely popular — every profile view, comment, and like on their account routes to the same shard. That one shard now handles far more load than the rest.

Two common fixes:

  • Compound shard keys — instead of hashing just user_id, hash something like user_id + a random suffix, spreading that one user's data across multiple shards.

  • A dedicated shard — detect unusually high-traffic keys and give them their own separate shard, sized to handle the extra load.

Have you ever noticed one part of a system getting disproportionately more traffic than everything else around it? That’s a hotspot, even outside the context of databases.

2. Cross-shard queries

Any query that needs data from more than one shard becomes more expensive. Instead of asking one database a question, you now have to ask several, wait for all of them, and combine the results yourself.

Query: "Top 10 posts across the platform"

App → Shard 1: get top posts → wait
App → Shard 2: get top posts → wait
App → Shard 3: get top posts → wait
App: combine all results, sort, return top 10

You can’t eliminate these entirely, but you can reduce how often they happen:

  • Pick a shard key that matches your most common queries in the first place

  • Cache the results of expensive cross-shard queries (e.g., refresh a “top posts” list every 5 minutes instead of computing it on every request)

  • Denormalize — duplicate some data across shards so a common query only needs to touch one shard, accepting that writes become slightly more work

If you find yourself constantly querying across all your shards for something routine, that’s usually a signal your shard key doesn’t match your actual usage pattern.

3. Consistency across shards

On a single database, a transaction — like moving money from one account to another — is atomic. Either both parts succeed, or neither does.

Once the two accounts live on different shards, that guarantee gets harder to maintain. You can’t rely on a single atomic transaction anymore.

Two common approaches:

  • Two-phase commit (2PC) — a coordinator checks that every shard involved is ready, then tells them all to commit at once. It works, but it’s slow, and if any shard or the coordinator fails mid-process, things can get stuck.

  • The Saga pattern — break the operation into smaller steps, where each step has a defined “undo” action if something later fails. For example: deduct money from account A, then add it to account B. If the second step fails, run a compensating action that refunds account A.

The general rule most systems follow: avoid transactions that span multiple shards wherever possible. When you truly can’t avoid it, the Saga pattern is usually a more practical choice than 2PC.

How to talk about sharding clearly

Whether you’re documenting a real system or explaining a design out loud, it helps to follow a simple, repeatable structure:

  1. State your shard key and why — based on your actual access pattern. (“Most queries fetch data for a single user, so we shard by user_id.")

  2. State your distribution strategy — usually hash-based with consistent hashing, unless there’s a specific reason otherwise.

  3. Acknowledge the trade-offs — which queries get harder, and how you plan to handle that (caching, denormalization, etc.).

  4. Explain how you’ll handle growth — how many shards you’re starting with, and how you’d add more later.

And before any of that — it’s worth asking whether you need to shard at all. A lot of systems never get close to the limits of a single, well-configured database. Doing the actual math (data size, expected writes per second, expected reads per second) and showing that you don’t need to shard yet can be just as valuable as knowing how to shard when you do.

Wrapping up

Sharding isn’t a magic scaling switch — it’s a trade-off. You gain the ability to scale storage and throughput horizontally, but you take on new complexity: choosing the right key, distributing data sensibly, and handling the operational headaches that come with spreading your data across machines.

The way I think about it: sharding should be the answer to a problem you’ve actually measured, not a default you reach for because it sounds advanced. Vertical scaling, caching, read replicas, and better indexing solve a lot of scaling problems on their own. Sharding is for when you’ve genuinely outgrown all of that.

One last question to leave you with: in the systems you’ve worked on, was sharding introduced because it was truly needed — or because it seemed like the “next step” everyone assumes you should take? I’d be curious to hear how that decision actually got made wherever you’ve seen it.

From Tech By Neha Gupta

  • 👏 Enjoyed the article? Don’t forget to leave a clap.

  • 💬 Have thoughts or questions? Share them in the comments.