Caching in System Design for Beginners
What caching actually does, where it fits in your system, and why adding a cache can create problems you didn’t expect.
Caching in System Design for Beginners
Imagine your application suddenly starts getting 100,000 requests every second.
Everything was working fine yesterday. Today, your database CPU is going crazy.
Queries are getting slower.
Users are waiting longer.
You add more application servers, but the database is still struggling.
Then someone says:
“Why don’t we add Redis?”
Sounds simple, right?
But here is the interesting part.
Redis can make your system much faster. But if you don’t understand caching properly, it can also introduce stale data, sudden database spikes, hot keys, and consistency problems.
So caching is not simply:
“Put Redis in front of the database.”
There is a lot more to it.
Let’s understand it from the beginning.
What is caching?
A cache is simply a temporary place where we keep frequently used data so we can access it faster later.
Think about something you use every day.
Suppose you frequently look at your bank balance.
Without caching, your application might do something like:
User
↓
Application Server
↓
Database
↓
Fetch balance
↓
Application
↓
UserNow imagine thousands of users doing the same thing.
Your database has to repeatedly fetch data that may not have changed.
Instead, we can keep frequently requested data in a faster storage layer:
User
↓
Application Server
↓
Cache
↓
Database (only when needed)The next time someone asks for the same data, we can get it from the cache.
This is the basic idea behind caching.
Why is a cache faster?
One important reason is where the data is stored.
Databases often need to work with storage such as SSDs. A cache like Redis commonly keeps data in memory (RAM).
Roughly speaking, the source material gives:
SSD access ≈ 1 millisecond
RAM access ≈ 100 nanosecondsThat’s a huge difference.
The exact numbers will vary depending on the hardware, workload, and system architecture, but the important idea is:
Memory is much faster to access than persistent storage.
And when your system is handling thousands or millions of requests, even a small latency difference can become important.
So caching makes a trade:
More memory + more complexity
↓
Less latency
Less DB load
More throughputBut where exactly should we put this cache?
Where can we put a cache?
Caching doesn’t have to exist in only one place.
There are several layers where caching can happen.
The most important ones are:
Client-side caching
CDN caching
In-process caching
External caching
Let’s understand each.
1. Client-side caching
This is probably the easiest one to understand.
The user’s own device stores some data.
For example, your browser may cache:
Images
CSS files
JavaScript files
HTTP responses
Instead of downloading the same image every time:
First request:
Browser → Server → Image
↓
Browser Cache
Later request:
Browser → Browser Cache → ImageNo request to your backend is necessary for that cached resource.
That’s extremely fast because the data is already on the user’s device.
Mobile applications can do something similar by storing data locally.
For example, a fitness app could store your recent running data on your phone so that you can still see it while offline.
The downside?
You don’t have complete control over the client’s cache.
The data might become stale.
And now you have to think about:
“Is the data on the user’s device still fresh?”
Client-side caching is useful, especially for offline or client-heavy applications, but it isn’t usually the first caching technique people discuss in a typical system design interview.
2. CDN caching
Now imagine your users are spread across the world.
Your server is located in Virginia.
One user is also in Virginia.
Another user is in Australia.
If the Australian user requests an image from your server, the request has to travel a very long distance.
That adds network latency.
A CDN solves this by keeping copies of content at servers located closer to users.
Think of it like this:
Origin Server
🇺🇸
|
┌──────────┼──────────┐
↓ ↓ ↓
CDN Edge CDN Edge CDN Edge
🇺🇸 🇪🇺 🇦🇺
↑ ↑
| |
US Users Australian UsersA user doesn’t always need to travel all the way to your origin server.
Instead:
User
↓
Nearest CDN
↓
Cached imageIf the CDN doesn’t have the image:
User
↓
CDN
↓
Origin Server
↓
CDN stores a copy
↓
UserThe next request can be served from the CDN.
CDNs are especially useful for:
Images
Videos
JavaScript files
CSS
Static files
Other frequently accessed content
Modern CDNs can also cache some API responses and run logic closer to users, but for system design interviews, media delivery is one of the easiest and most common use cases to explain.
3. In-process caching
Now let’s move inside your application server.
Suppose you have a Node.js application.
You can keep some data directly inside the application’s memory.
For example:
const cache = new Map();
cache.set("config", {
maxRetries: 3,
timeout: 5000
});When you need it:
const config = cache.get("config");There is no Redis request.
There is no network call.
The data is already inside your application.
That’s extremely fast.
But there is an important problem.
Imagine you have three application servers:
Load Balancer
/ | \
/ | \
↓ ↓ ↓
Server A Server B Server C
↓ ↓ ↓
Local Local Local
Cache Cache CacheIf Server A stores:
user:123 → NehaServer B doesn’t automatically know about it.
Each server has its own memory.
This can lead to:
duplicated data
wasted memory
different servers having different versions of the data
So in a large distributed system, an external cache is often more convenient.
4. External caching
This is where tools such as Redis or Memcached come in.
Instead of each application server having its own cache, we create a shared caching layer.
Load Balancer
/ | \
↓ ↓ ↓
Server A Server B Server C
\ | /
\ | /
↓ ↓ ↓
Redis
↓
DatabaseNow all application servers can use the same cache.
If Server A fetches some data and stores it in Redis, Server B can reuse it.
This is one of the most common caching architectures you’ll see in system design.
Cache Hit vs Cache Miss
Before going further, you need to understand two words.
Cache Hit
The requested data is already in the cache.
Application
↓
Cache
↓
Found ✅
↓
Return dataThis is fast.
Cache Miss
The requested data isn’t in the cache.
Application
↓
Cache
↓
Not found ❌
↓
Database
↓
Store in Cache
↓
Return dataUnderstanding hit vs miss is important because almost every caching design is built around this idea.
Cache-Aside: The Pattern You Should Know First
There are different ways an application can interact with a cache.
But if you’re just starting, learn cache-aside first.
It’s simple and extremely useful.
The application checks the cache before going to the database.
Request
↓
Check the cache
/ \
HIT MISS
↓ ↓
Return Database
data ↓
Store in
Cache
↓
Return dataLet’s see some code.
Imagine we want to fetch a user.
async function getUser(userId) {
const key = `user:${userId}`;
// 1. Check cache first
const cachedUser = await redis.get(key);
if (cachedUser) {
return JSON.parse(cachedUser);
}
// 2. Cache miss → fetch from database
const user = await db.users.findById(userId);
// 3. Store the result for future requests
await redis.set(
key,
JSON.stringify(user),
"EX",
300 // expire after 5 minutes
);
return user;
}The first request might look like:
Request
↓
Redis ❌
↓
Database
↓
Redis ✅
↓
ResponseThe next request:
Request
↓
Redis ✅
↓
ResponseThe database doesn’t need to do the work again.
That’s the basic power of caching.
Why not cache everything?
This is a very important question.
If caching is so fast, why not put the entire database in Redis?
Because a cache isn’t free.
Memory is limited.
And cached data needs to be managed.
Suppose your application has:
1 billion usersbut only:
1 million usersare actively being requested.
Caching all 1 billion users may waste a huge amount of memory.
With cache-aside, you can cache only what people actually request.
That’s one of its biggest advantages.
Other Cache Architectures
Cache-aside isn’t the only approach.
There are three other terms worth knowing:
Write-through
Write-behind
Read-through
You don’t necessarily need to memorize these names.
What matters more is understanding who reads/writes what and when.
Write-through caching
With write-through caching, the application writes to the cache, and the cache synchronously writes to the database.
Application
↓
Cache
↓
DatabaseThe write isn’t considered complete until both are updated.
This can help keep the cache and database aligned.
But writes become slower because the system has to update both.
It can also put data into the cache that may never actually be read.
And there’s another difficult problem.
What happens if:
Cache update ✅
Database update ❌Now the two systems disagree.
This is often called the dual-write problem.
Write-behind caching
Write-behind, also called write-back, takes this idea further.
The application writes to the cache first.
The cache updates the database later, asynchronously.
Application
↓
Cache
↓
Later...
↓
DatabaseThis can make writes very fast.
But there is a trade-off.
What if the cache crashes before the data reaches the database?
You could lose data.
So this approach makes more sense when very high write throughput is more important than immediate consistency and some data loss may be acceptable, such as certain analytics or metrics workloads.
Read-through caching
Read-through is similar to cache-aside.
The big difference is:
The cache itself handles the database lookup on a miss.
Application
↓
Cache
↓
MISS
↓
Database
↓
Cache
↓
ApplicationYou can think of the cache as acting like a proxy.
This pattern is particularly common when talking about CDN or edge caching.
For most application-level caching, cache-aside is still a simple default because the application can explicitly control the flow.
Don’t memorize the names
This is worth repeating.
If you forget the term cache-aside during an interview, don’t panic.
You can simply say:
“I’ll check Redis first. If the data isn’t there, I’ll query the database, store the result in Redis, and return it.”
That explanation is much more useful than remembering four names but not understanding how they work.
What happens when the cache becomes full?
Here’s another problem.
Memory is limited.
Suppose your cache can store only three items:
[A] [B] [C]Now you need to store:
[D]Something has to go.
But which one?
That’s where eviction policies come in.
An eviction policy decides which cached item should be removed when the cache needs space.
1. LRU — Least Recently Used
LRU removes the item that hasn’t been used recently.
Imagine:
A → used 1 minute ago
B → used 10 seconds ago
C → used 2 hours agoIf we need space, C is a good candidate for removal.
The thinking is:
“If nobody has used it recently, maybe nobody needs it right now.”
LRU is one of the most common policies to know for system design interviews.
2. LFU — Least Frequently Used
LFU looks at how often something is accessed.
For example:
A → accessed 1,000 times
B → accessed 500 times
C → accessed 2 timesC becomes a candidate for eviction.
This can be useful when a small number of items are accessed much more frequently than the rest.
3. FIFO — First In, First Out
This one is simple.
The oldest item goes first.
[A] [B] [C]
↑
oldestAdd D:
[B] [C] [D]A is removed.
It’s easy to understand but isn’t always the best choice for caching.
4. TTL — Time To Live
TTL says:
“Keep this data for a certain amount of time.”
For example:
user:123
TTL = 5 minutesAfter five minutes, the cache entry expires.
TTL is useful when data can become stale.
For example:
News feed → TTL 60 seconds
Exchange rate → TTL 30 seconds
User profile → TTL 5 minutesThe correct TTL depends completely on the system.
Now Things Get Interesting
At this point caching probably sounds great.
You put frequently used data into memory.
Your database receives fewer requests.
Your application becomes faster.
Done?
Not quite.
Caching creates a second copy of your data.
And the moment you have two copies, you have new problems.
This is why cache design is interesting.
Let’s look at the three problems you should know.
Problem 1: Cache Stampede
Imagine your homepage is cached for 60 seconds.
Your system gets:
100,000 requests / secondEverything is fine.
All those requests hit the cache.
Then suddenly:
60 seconds passThe cache entry expires.
Now what happens?
All 100,000 requests can see:
CACHE MISSSo they all go to the database.
100,000 requests
↓
Cache expired
↓
┌──────────┼──────────┐
↓ ↓ ↓
DB DB DB
\ | /
\ | /
└────────┼────────┘
↓
Database overloadInstead of the cache protecting your database, you’ve suddenly sent a huge wave of requests directly to it.
This is called a cache stampede or thundering herd.
And yes, it can take down a database.
How do we prevent a cache stampede?
Approach 1: Request coalescing
Suppose 1,000 requests all need the same missing key.
Don’t let all 1,000 requests query the database.
Let one request rebuild the cache.
The others wait.
Request 1 ──→ Database ──→ Cache
Request 2 ────────────────→ Wait
Request 3 ────────────────→ Wait
Request 4 ────────────────→ WaitOnce Request 1 finishes:
Cache
↓
All waiting requestsThis is often called single flight or request coalescing.
Approach 2: Cache warming
Instead of waiting for the cache to expire completely, refresh popular entries before expiration.
For example:
TTL = 60 seconds
At 55 seconds:
Refresh the cacheSo the entry gets another 60 seconds.
The key never reaches a point where thousands of users discover an empty cache simultaneously.
Problem 2: Cache Consistency
Here’s one of the most common caching questions.
Imagine a social media application.
A user changes their profile picture.
The database now contains:
profilePicture = image-2But the cache still contains:
profilePicture = image-1Now another user requests the profile.
They hit the cache.
They see:
image-1even though the database says:
image-2So which one is correct?
This is the cache consistency problem.
One solution: Invalidate on write
When updating the database, remove the corresponding cache entry.
Update profile
↓
Update database
↓
Delete cache keyThen the next request gets:
Cache miss
↓
Database
↓
Fresh data
↓
CacheThis is called cache invalidation.
It’s a common approach when stale data is not acceptable.
Another solution: Short TTL
Sometimes stale data isn’t a big deal.
Imagine a news feed.
If someone sees a feed that is 30 seconds old, that’s probably acceptable.
In that case:
TTL = 30 secondscan be enough.
After the TTL expires, the cache is rebuilt.
The key question is:
How stale can my data safely be?
That question should drive your caching strategy.
Eventual consistency isn’t always a problem
This is an important mindset shift.
Not every system needs every user to see the latest data immediately.
For example:
Analytics dashboard
A few seconds of delay?
Probably okay.
News feed
A minute of delay?
Potentially okay.
Bank balance
Showing an outdated balance?
Much more problematic.
So don’t automatically try to make every cache perfectly consistent.
Instead ask:
What happens if the user sees stale data for 30 seconds?
If the answer is “nothing serious,” eventual consistency may be completely reasonable.
Problem 3: Hot Keys
Now imagine you have a huge distributed Redis cluster.
Everything looks healthy.
But suddenly everyone wants the same piece of data.
For example:
Taylor Swift's profileMaybe millions of requests arrive for:
user:taylor-swiftIf that key lives on one Redis node, that node can become overloaded.
Millions of requests
↓
user:taylor-swift
↓
Redis Node
💥Notice something interesting:
Your overall cache can be healthy while one key is killing a node.
That’s a hot key.
How can we handle hot keys?
Replicate the key
Instead of storing the popular key on only one cache instance:
Redis 1 → Taylor Swift
Redis 2 → Taylor Swift
Redis 3 → Taylor SwiftRequests can now be distributed across multiple instances.
Taylor Swift
↓
┌────────┼────────┐
↓ ↓ ↓
Redis 1 Redis 2 Redis 3Use a local cache
You can also keep extremely popular data in the application’s own memory.
Request
↓
Local Cache
↓
Redis
↓
DatabaseIf the value is extremely hot, repeated requests may never need to reach Redis.
This is where the earlier idea of in-process caching becomes useful again.
The Big Lesson
At this point, you can see something important.
Caching isn’t simply about making reads faster.
It’s a balancing act:
CACHING
│
┌───────────┼───────────┐
↓ ↓ ↓
Speed Database Complexity
Load
│
↓
New Problems
│
┌───────────┼───────────┐
↓ ↓ ↓
Stampede Consistency Hot KeysAnd this is exactly why caching becomes an important topic in system design interviews.
When should you actually add a cache?
Here’s a mistake I see people make when learning system design:
They draw Redis immediately.
API → Redis → DatabaseAnd when asked:
“Why Redis?”
They say:
“Because Redis is fast.”
That’s not enough.
A cache should solve a problem.
There are a few common reasons to introduce one.
1. Your system is read-heavy
Suppose your database is receiving millions of reads.
But the same data is being requested again and again.
Caching can remove a large portion of those database reads.
Without cache:
1,000,000 requests
↓
1,000,000 DB readsWith a good cache:
1,000,000 requests
↓
900,000 cache hits
+
100,000 DB readsThe exact numbers will depend on the workload, but the idea is simple:
Let the cache absorb repeated reads.
2. The query is expensive
Imagine generating a personalized news feed requires:
Posts
+
Followers
+
Likes
+
Comments
+
Ranking
+
Multiple database queriesDoing all that work for every request can be expensive.
Instead, you could cache the generated feed for a short period.
Generate feed
↓
Store in Redis
↓
Serve repeatedly
↓
Refresh after TTLNow you aren’t rebuilding the same expensive result every time.
3. You have strict latency requirements
Suppose your API requirement is:
Response time < 100 msBut your database operation is expensive enough that you’re struggling to meet that requirement.
A cache can help by serving frequently requested data from a faster layer.
4. Your database is becoming the bottleneck
In real systems, you have metrics.
Maybe you discover:
Database CPU → 90%
Read traffic → extremely high
Cacheable requests → hugeThat is a strong signal that caching might help.
In a system design interview, you won’t have real production metrics, so you’ll need to reason using rough numbers.
A Simple Framework for System Design Interviews
When you’re designing a system and thinking about caching, don’t immediately draw Redis.
Ask yourself these questions.
Question 1: What is my bottleneck?
Is it:
Database reads?
Expensive computation?
Latency?
Database CPU?
If you can’t identify a problem, why are you adding a cache?
Question 2: What exactly am I caching?
Don’t say:
“I’ll cache the users.”
Be specific.
For example:
Key:
user:123
Value:
{
"name": "Neha",
"profilePicture": "...",
"followers": 1200
}Or:
Key:
feed:user:123
Value:
List of posts for the user's feedThe cache key matters.
Question 3: How will I read from the cache?
For example:
Request
↓
Redis
↓
HIT? → Return
↓
MISS
↓
Database
↓
Redis
↓
ReturnFor many application-level systems, cache-aside is a straightforward choice.
Question 4: How long should the data stay?
Maybe:
TTL = 30 secondsor:
TTL = 5 minutesor perhaps you use LRU/LFU depending on the access pattern.
But don’t just mention a policy.
Explain why.
For example:
“The feed changes frequently, but a few seconds of stale data is acceptable, so I’ll use a short TTL.”
That is much stronger than:
“I’ll use TTL.”
Question 5: What happens when things go wrong?
This is where a good system design answer becomes much better.
Ask:
What if the cache expires?
Could we get a cache stampede?
What if the cache contains stale data?
Is that acceptable?
What if one key gets millions of requests?
Could we have a hot key?
What if Redis goes down?
Can the application fall back to the database?
What if the cache and database disagree?
How will we handle invalidation?
These questions show that you’re thinking about the system beyond the happy path.
A Small Interview Example
Suppose you’re asked:
“Design a news feed.”
You might start with:
User
↓
API Server
↓
DatabaseThen you realize:
Users request their feed frequently.
Generating the feed is expensive.
A feed can tolerate a little staleness.
You need low response latency.
Now caching has a reason to exist.
You could say:
“Since generating the feed requires expensive computation and the same feed may be requested repeatedly, I’ll cache the generated feed in Redis using cache-aside.”
Then:
User
↓
API Server
↓
Redis
↓
├── HIT → Feed
│
└── MISS
↓
Generate Feed
↓
Database
↓
Redis
↓
UserThen you can add:
“I’ll use a short TTL because the feed changes frequently, and some eventual consistency is acceptable.”
Now you’re not just adding Redis.
You’re explaining why it exists.
The Mental Model I Use for Caching
If I had to reduce this entire topic to one thought process, it would be:
START
↓
Is something slow?
↓
YES
↓
Is the data requested
repeatedly?
↓
YES
↓
Is it okay to keep a
temporary copy?
↓
YES
↓
CACHE
↓
┌───────────┼───────────┐
↓ ↓ ↓
What data? How long? What can
go wrong?
↓
┌──────────┼──────────┐
↓ ↓ ↓
Stampede Stale data Hot keyThat’s the real skill.
Not memorizing Redis commands.
Not memorizing four caching patterns.
Not drawing a cache box in every architecture.
The skill is knowing when caching solves a real problem and understanding the trade-offs it introduces.
Final Thoughts
When I first learned system design, caching felt almost too easy.
The explanation seemed to be:
Database is slow → Redis is fast → add Redis.
But once you start thinking about real systems, the interesting questions begin.
What if the cache is full?
What if the cached data is stale?
What if the cache expires at exactly the wrong time?
What if one key becomes incredibly popular?
What if Redis goes down?
What if the database and cache disagree?
That’s why caching is much more than a performance trick.
It’s another component in your distributed system.
And every additional component brings both benefits and failure modes.
So the next time you see a system design problem, don’t start by drawing Redis.
Start with the problem.
What is slow?
What is expensive?
What data is being requested repeatedly?
How fresh does that data need to be?
What happens when the cache fails?
Once you start asking these questions, caching becomes much easier to reason about.
And if you’re preparing for system design interviews, remember one thing:
Don’t try to impress the interviewer by saying “I’ll use Redis.”
Impress them by explaining why you need it, what you’re caching, and what happens when it goes wrong.
That’s the difference between knowing caching and being able to design with caching.