I wanted to cache a query. Every answer created a new question.

This article isn't about Redis.

It isn't about cache invalidation.

It isn't about TTLs or distributed locks.

At least, not at first.

The story begins with a single database query.

Along the way, we'll encounter caching, cache expiration, cache stampedes, request coalescing, horizontal scaling, Redis, and distributed coordination.

But none of those technologies appear because we decided to use them.

They appear because the previous solution stopped being enough.

One question leads to another.

One optimization exposes a new limitation.

One architectural decision changes the problem we are trying to solve.

By the end of the article, my hope is that you'll understand something much more valuable than how caching works.

You'll understand why experienced engineers often arrive at similar architectures—not because they follow the same tutorials, but because they're responding to the same constraints.

If you've ever wondered why so many production systems seem to converge toward the same ideas, this is one possible path.

Here's the journey we're about to take.

Database Query
       │
       ▼
Duplicated Work
       │
       ▼
Cache
       │
       ▼
Freshness
       │
       ▼
TTL
       │
       ▼
Cache Stampede
       │
       ▼
Request Coalescing
       │
       ▼
Multiple Servers
       │
       ▼
Shared State
       │
       ▼
Redis
       │
       ▼
Shared Failures

Don't worry if some of these concepts are unfamiliar.

By the end of the article, each one will feel like the natural consequence of the previous one.

The problem

Imagine the following code:

const result = await searchInDb();

return result;

There is nothing particularly interesting about it. The query is properly indexed, the database is healthy, and the response time is acceptable. If this code appeared in a pull request, there would probably be little reason to question it.

Now suppose that one million users request exactly the same data at approximately the same time.

I am deliberately using an extreme number. Most applications will never receive one million simultaneous requests for the same resource, and the first question should always be whether the existing database can already handle the expected load. If it can, introducing additional infrastructure may solve a problem that does not actually exist.

The extreme scenario is useful because it makes one particular property of the workload easy to see: every request is asking the system to perform exactly the same work.

Before continuing, let us make a few assumptions about the system:

• Every request produces the same result.
• The result changes relatively infrequently.
• Returning a slightly outdated value is acceptable.
• The database eventually becomes a bottleneck because it repeatedly performs the same work.
• We will gradually increase the scale of the application and reconsider the design whenever one of these assumptions changes.

These assumptions are intentionally restrictive. A query that returns personalized data, a financial balance, or information that must always be current may require a completely different design. The purpose of the example is not to find a universally correct architecture, but to examine how architectural decisions follow from a particular set of constraints.

At first, the obvious response is to scale the database. If the database cannot handle the traffic, perhaps it simply needs more capacity. There are several reasonable ways to obtain it:

• Improve the indexes.
• Rewrite or optimize the query.
• Add more CPU or memory.
• Introduce read replicas.
• Precompute part of the result.
• Limit the number of requests that may execute concurrently.
• Apply optimizations specific to the database being used.

Any of these approaches may be sufficient. In a real system, this is where measurement matters: query plans, latency distributions, database utilization, lock contention, and the expected request rate should inform the decision.

However, all of these options share an interesting property. They make the database better at performing the work, but they do not question whether the work needs to be performed repeatedly.

If one million requests ask the same question, a faster database may answer that question one million times more efficiently. A read replica may distribute those executions across several machines. A concurrency limit may prevent the system from becoming overloaded. None of them changes the fact that the same result is being computed again and again.

This suggests that the database may not be the only problem. The more fundamental problem may be duplicated work.

The next question is therefore not how to execute the query faster, but whether every request needs to execute it at all.

Reusing previous work

The previous section ended with a different question.

Instead of asking how to make the database faster, we started asking whether every request actually needs to execute the query.

At first glance, the answer seems obvious: of course it does. Every HTTP request reaches our application independently, so every request naturally executes the same code.

const result = await searchInDb();

return result;

But if we stop thinking about requests for a moment and focus on the computation itself, something interesting appears.

Suppose the first request executes the query and produces the result.

A second request arrives one second later asking exactly the same question.

Should the application really execute the same query again?

The answer depends on something we haven't discussed yet.

Has the data changed?

If nothing has changed since the first request, executing the query again will produce exactly the same result. The database is not generating new information—it is simply recomputing something the application already knows.

In other words, the second request is not asking for new work.

It is asking for previously computed work.

Once we notice this, a much simpler question emerges.

If we already know the answer, why are we asking the database again?

One possible solution would be to remember the result produced by the first request and reuse it whenever another request asks the same question.

Conceptually, the application would behave like this:

This idea is surprisingly simple. Rather than making the database execute the same computation repeatedly, the application remembers the previous answer and reuses it whenever it is still valid.

This technique has a name.

It is called caching.

Caching isn't a single technology. It's an idea: reusing previous work instead of repeating it.

That idea can be implemented in many places. A browser can cache responses. A CDN can cache content close to users. A framework may cache expensive computations. A database may cache pages in memory.

Throughout this article, we'll focus on application-level caching because it makes the architectural questions easier to see.

Notice how we arrived here. We did not start by asking, "Should we use Redis?" or "Should we add a cache?" We simply observed that the application was performing identical work to produce identical results. Once that property became visible, remembering previous results became a natural consequence rather than a design decision.

Of course, introducing a cache immediately raises another question.

How do we know whether a previously computed result is still valid?

When does the answer stop being correct?

Reusing previous work sounds almost perfect.

If computing the result is expensive and the answer hasn't changed, returning the previous result avoids unnecessary work and dramatically reduces the load on the database.

Unfortunately, this immediately introduces a new problem.

The database is no longer the only place where the information exists.

The application now remembers previous results, while the database continues to evolve independently.

Imagine a product catalog.

At 10:00 AM, a user requests the list of products.

The application executes the query, stores the result, and returns it.

A few minutes later, someone adds a new product to the database.

Now another user requests exactly the same endpoint.

Should we return the previous result?

Not anymore.

The previous answer is no longer correct.

This reveals something important.

Whether we can reuse a previous result has nothing to do with the request itself.

The request is identical.

The question is whether the underlying data has changed.

In other words, a cached value is only useful while it continues to represent reality.

That naturally leads to another question.

How should the application determine whether a previously computed result is still valid?

There are several possible strategies:

• Detect every change in the underlying data.
• Associate a version with the data.
• Explicitly invalidate previous results after updates.
• Assume the result remains valid for a fixed period of time.

Each strategy makes different trade-offs. Some maximize freshness, while others prioritize simplicity or performance. The correct choice depends entirely on the guarantees the system needs to provide.

Some systems never let cached values expire automatically. Instead, they explicitly invalidate cached values whenever the underlying data changes.

For many applications, however, the simplest answer is surprisingly effective.

Rather than trying to determine exactly when the data changes, we simply assume that the cached value remains valid for a limited amount of time.

After that period expires, the application recomputes the value and stores a new one.

This is known as a Time To Live (TTL).

TTL is attractive because it requires very little coordination. The application does not need to know why the data changed; it simply assumes that after some amount of time it should no longer be trusted.

That simplicity is also its limitation.

At first, this seems to solve the problem.

The application avoids repeating expensive work while periodically refreshing the stored value.

But, as before, solving one problem quietly creates another.

What happens when that TTL expires and one million requests arrive at exactly the same moment?

When everyone discovers the expiration at the same time

At this point, our system looks much healthier than before.

Most requests never reach the database because they reuse a previously computed result. Only when the cached value expires do we execute the query again.

For a while, everything works exactly as expected.

Then the cache expires.

Normally, that is not a problem.

The first request notices that the stored value is no longer valid, executes the query again, stores the fresh result, and the following requests reuse it.

But remember the scenario we started with.

One million users requesting exactly the same resource.

What happens if the cached value expires one millisecond before those one million requests arrive?

Every request reaches the application.

Every request checks whether a previous result exists.

Every request reaches exactly the same conclusion.

"There is no valid result."

None of them is wrong.

The cache really has expired.

The problem is that they all discover it simultaneously.

Once that happens, every request behaves exactly as we programmed it to behave.

Each one independently executes the expensive query.

Ironically, the cache has not failed.

It behaved exactly as designed.

Every request correctly detected that the stored value had expired.

The real problem is that expiration does not happen gradually.

Every request observes the same expiration at almost exactly the same time, and every one of them independently decides to rebuild the same value.

The duplicated work that caching eliminated has suddenly returned.

Only now it happens during a very short period of time, producing a massive spike of traffic against the database.

This problem has a name.

It is known as Cache Stampede.

Notice how we arrived here.

TTL was not a mistake.

Caching was not a mistake.

Neither mechanism is broken.

The problem appears because independently behaving requests suddenly need to coordinate with each other.

Until now, every request could make decisions on its own.

For the first time in the article, that assumption is no longer true.

If rebuilding the cached value is expensive, perhaps only one request should perform that work while everyone else waits for the result.

Only one request should rebuild the cache

At the end of the previous section, we reached an important conclusion.

The problem was never that the cache expired.

The problem was that every request independently decided to rebuild the same value.

If rebuilding the cached value is expensive, allowing one million requests to execute the same query simultaneously defeats the entire purpose of having a cache.

So perhaps we should change the rules.

Instead of allowing every request to rebuild the value, what if only the first request were allowed to do so?

Every other request could simply wait.

Conceptually, the system would behave like this.

Now imagine what happens.

The first request discovers that the cache has expired.

It starts executing the expensive query.

A second request arrives a few milliseconds later.

Previously, it would have executed exactly the same query.

Now it notices that someone else is already rebuilding the value.

Instead of repeating the work, it waits.

The third request behaves the same way.

So does the fourth.

And eventually, thousands—or even millions—of requests are all waiting for the exact same computation to finish.

Nothing new is being computed.

Everyone is simply waiting for the first request to finish the work they all need.

Something interesting has happened.

The expensive operation has become a shared resource.

The application is no longer coordinating access to the cached value.

It is coordinating access to the computation itself.

Once the first request finishes, every waiting request immediately receives the freshly computed result.

Notice what changed.

Originally, one million requests produced one million database queries.

With caching, one million requests usually produced zero database queries—but occasionally produced one million again when the cache expired.

Now, one million simultaneous requests produce exactly one database query.

The rest simply wait for its result.

This technique is commonly known as Request Coalescing, Single Flight, or Duplicate Suppression, depending on the language, framework, or library being used.

The name is far less important than the underlying idea.

Multiple requests are not competing for data.

They are competing to perform the exact same work.

By allowing only one of them to execute that work, the application eliminates an enormous amount of unnecessary computation.

At first glance, this seems to solve the problem completely.

But there is one assumption hidden in everything we have built so far.

Every request reaches the same application instance.

The solution worked. Until we added another server.

So far, every request has reached the same application instance.

That detail may have gone unnoticed because, until now, it did not matter.

Every request shared the same memory.

Every request could see the same cache.

Every request could also see whether another request was already rebuilding a value.

As long as the application runs as a single process, coordinating requests is relatively straightforward.

Then traffic grows.

One server is no longer enough.

We deploy a second application instance behind a load balancer.

Later, a third.

Then five.

Then twenty.

At first, this feels like a pure infrastructure improvement.

The application code hasn't changed.

The cache still exists.

The request coalescing logic still exists.

Everything should continue working.

Except it doesn't.

Imagine that the cached value expires.

Three requests arrive almost simultaneously.

The load balancer distributes them across three different servers.

Each backend checks its local cache.

Each backend reaches exactly the same conclusion.

"There is no valid value."

Each backend also checks whether another request is already rebuilding it.

And each backend reaches exactly the same conclusion again.

"No."

None of them is lying.

Each server is only looking at its own memory.

Backend A has absolutely no idea what Backend B is doing.

Backend B cannot see Backend C.

From the perspective of each process, it is completely alone.

Our request coalescing logic is still working perfectly.

Within each server.

The problem is that our application is no longer one process.

It is several independent processes that happen to serve the same users.

Without realizing it, we changed one of the assumptions that every previous solution depended on.

Memory is no longer shared.

And once memory stops being shared, coordination disappears with it.

This changes the question once again.

The problem is no longer how to coordinate requests.

The problem is how to coordinate servers.

Somewhere, there needs to be a place that every backend can see.

A place where they all agree on:

• whether a cached value already exists,
• whether someone is currently rebuilding it,
• and what the latest result is.

In other words, the cache itself can no longer live inside a single application's memory.

It must live somewhere that every application instance can access.

Only now do we finally have a reason to introduce another component into the architecture.

Not because Redis is "the cache everyone uses."

But because our cache has become a shared resource, and shared resources require shared state.

A shared memory for independent servers

Our application has become distributed.

Every backend is capable of serving requests independently, but they all need to agree on the same state.

If one server is already rebuilding a cached value, every other server should know about it.

If one server has already computed the latest result, every other server should be able to reuse it.

In other words, every backend needs access to the same memory.

Of course, the operating system cannot magically merge the memory of several independent processes running on different machines.

That shared memory has to live somewhere else.

Notice what has changed.

Originally, memory was an implementation detail.

Each application simply stored values inside its own process.

Now memory has become part of the architecture itself.

It is no longer enough to ask whether a value exists.

Every server must ask the same place.

The same is true for request coalescing.

Previously, a request only needed to know whether another request in the same process was rebuilding the value.

Now it needs to know whether any server in the system is rebuilding it.

The coordination has become global.

At this point, we finally have a concrete set of requirements.

We need something that allows every backend to:

• read previously computed values,
• write new values,
• coordinate rebuilds,
• respond extremely quickly,
• and support thousands of concurrent operations.

Notice what we still have not done.

We have not chosen a technology.

We have only described the properties the system requires.

Many systems can satisfy some or all of these requirements.

Distributed caches, managed cloud services, API gateways, or even databases themselves can all provide different forms of shared state depending on the problem being solved.

Redis is one particularly popular choice because it matches these requirements remarkably well.

Redis is often described as an in-memory database.

While technically accurate, that description misses why it appears so frequently in distributed systems.

In our case, Redis is not interesting because it stores key-value pairs.

It is interesting because it gives independent servers a place where they can coordinate.

The cache no longer belongs to Backend A.

Or Backend B.

Or Backend C.

It belongs to the entire system.

The same is true for the information that someone is currently rebuilding a value.

That information also becomes shared.

The architecture has crossed an important boundary.

Until now, every optimization relied on local memory.

From this point forward, every optimization relies on shared state.

And that introduces an entirely new class of questions.

Shared state introduces shared failure

Moving the cache to Redis solved one important problem.

Every application instance now reads and writes the same cached value.

If Backend A rebuilds the cache, Backend B immediately sees the new result.

If Backend C wants to know whether someone is already rebuilding the value, it can ask the same shared system.

The coordination problem is no longer local.

It has become distributed.

At first, this feels like the final architecture.

But something subtle has happened.

Earlier in the article, every problem originated inside the application itself.

Now, an entirely new component has become part of every request.

Every cache lookup depends on Redis.

Every cache write depends on Redis.

Every rebuild lock depends on Redis.

Redis is no longer an optimization.

It has become critical infrastructure.

That naturally leads to another question.

What happens if Redis is unavailable?

There is no single answer.

Different systems make different trade-offs.

Some applications bypass the cache entirely and read directly from the database.

Others continue serving previously cached values for a limited amount of time.

Some reject the request altogether because returning outdated information would be unacceptable.

The important observation is not which strategy is correct.

It is that introducing shared state also introduces shared failure.

A local cache failure affects one process.

A shared cache failure affects the entire system.

The architecture has become more powerful, but also more interconnected.

Every architectural decision from this point forward is about balancing competing goals.

• Freshness versus availability.
• Simplicity versus coordination.
• Latency versus consistency.
• Resource usage versus correctness.

None of these trade-offs has a universally correct answer.

The requirements of the system determine which compromises are acceptable.

And that brings us back to the question that started this entire article.

We wanted to cache a query.

What we actually discovered was that every answer created a new question.

Every answer created a new question

When this article began, the system consisted of a single database query.

const result = await searchInDb();

return result;

Nothing looked particularly complicated.

There was no Redis.

No distributed locks.

No load balancers.

No cache invalidation strategies.

No request coalescing.

Just a query.

As we explored the problem, each solution solved something important.

Caching eliminated duplicated work.

TTL prevented results from living forever.

Request coalescing prevented thousands of identical computations.

Redis allowed independent servers to coordinate.

None of these ideas were wrong.

In fact, every one of them was the right solution to the problem that existed at that moment.

The interesting part is what happened next.

Every solution quietly changed the architecture.

And every architectural change revealed a new limitation that had previously been invisible.

Caching made us think about freshness.

Freshness made us think about expiration.

Expiration made us think about concurrent requests.

Concurrent requests made us think about coordination.

Coordination made us think about distributed systems.

Distributed systems made us think about shared state.

Shared state made us think about failure.

At no point did we wake up and decide to build a distributed system.

We simply kept asking the next question.

This is why system design is often difficult to teach by starting with technologies.

If someone had introduced Redis in the first paragraph, it would have sounded like an arbitrary architectural choice.

By the time we reached Redis ourselves, the need for shared state no longer felt optional.

Redis was simply one possible implementation of that requirement.

The same is true for every other concept in this article.

We did not begin by searching for opportunities to use caches, TTLs, distributed locks, or shared state.

We began with a simple observation.

One million users were asking exactly the same question.

Everything else followed from that.

Good software architecture rarely emerges from knowing every available technology.

It emerges from understanding the current constraints, solving today's problem, and recognizing the new questions that every solution inevitably creates.

That's why this article was never really about caching.

It was about learning how to think.

Because in software engineering, every answer creates a new question.