Intermediate System design concept · Handling Scale & Bottlenecks · 32 mins read

Caching Systems

Caching saves expensive work by serving a fast copy closer to the request path.

Open the interactive lesson See it in Design Library Management System

Cache-Aside

Learn the default caching pattern where the application loads on a miss, writes the database directly, and invalidates stale keys on change.

Intuition

The database is correct but slow compared with memory. If every read goes to the database, hot keys repeatedly pay network and storage latency even though the answer often has not changed. Cache-aside solves that by reading from cache first and only touching the database on misses. Cache-aside is the pattern interviewers expect as the default answer because it is explicit, easy to add incrementally, and fails gracefully: if Redis is down, the application can still read from the database. The hard part is not the happy path; it is invalidation, race conditions, and protecting the database when a hot key expires.

Mental Model

Treat the cache as a fast copy, not the source of truth. The app checks the cache; on a hit it returns immediately, and on a miss it fetches from the database, stores that value in cache, and serves the result. Writes usually go straight to the database, then the app deletes or updates the cache entry so the next read reloads the newest value. Think of it like: Think of a store clerk keeping a notepad of the most requested prices. If a price is on the notepad, the clerk answers instantly. If it is missing, the clerk walks to the warehouse ledger, copies the current price to the notepad, and answers. When headquarters changes a price, the clerk crosses out the old note so the next lookup copies the fresh value.

Building Blocks

Cache-First Read Path: The app checks cache before the database. On a miss it loads from the source of truth, stores the value, and returns it.
Invalidate After Writes: After the DB commit, related cache keys are deleted or refreshed. Delete-on-write is common because it avoids trying to update every derived key in place.
Miss-Storm Protection: Hot-key expiry can send many identical reads to the database. Single-flight refill, jitter, or refresh-ahead keep one expired key from becoming a spike.

Definitions

Cache-Aside

A pattern where the application repopulates cache only after a miss.

Reads are cache-first; writes usually go to the database directly.
The app owns keys, refill logic, and invalidation.

Invalidation

Removing or refreshing cached entries after the underlying data changes.

Delete-on-write is the safest common variant.
Missed invalidation is the main source of stale cache-aside data.

Stale Read

A cached response that no longer matches the latest committed database value.

TTL limits how long stale data can survive, but does not prevent it.
Whether staleness is acceptable depends on the product.

Patterns

Delete on Write — When the database is authoritative and cached objects have multiple derived views that are hard to update in place.
Negative Caching — When repeated lookups for missing objects are expensive or abusive.

Strategies

Lazy Load Only on Miss When: As the default for read-heavy data that does not need to be prewarmed. How: Check cache first, fetch from the database only on a miss, then store the result with a TTL. The cache stays focused on data traffic actually reuses. Example: A product endpoint reads product:123 from Redis if present; otherwise it reads PostgreSQL, caches the JSON, and returns it.
TTL Plus Coalesced Refill When: When invalidation is mostly correct but hot-key expiry still needs protection. How: Use TTL as a safety net, add jitter so keys do not expire together, and coalesce refill for very hot keys. Example: A feed service adds TTL jitter and allows only one worker to rebuild a trending-page key at a time.

The stale repopulation race

The easy version of cache-aside is “write DB, delete cache, done.” The subtle bug appears when a reader fetched the old database value just before a concurrent writer committed a new value. If that reader repopulates cache after the writer already invalidated it, the cache becomes stale again even though the invalidation happened.

That is why experienced engineers treat cache-aside as a concurrency problem, not just a storage shortcut. Common fixes are delete-after-commit, value versions or updated_at checks on repopulation, and single-flight refill on very hot keys.

Tradeoffs

Decision	Upside	Downside
Application control vs application complexity	You can choose key shape, TTL, and invalidation per object type, which makes cache-aside adaptable to mixed workloads.	Correctness lives in app code, so every service must implement refill, invalidation, and stampede protection carefully.
Delete-on-write freshness vs post-write cache misses	Deleting after commit is simple and keeps the next fill tied to the source of truth.	The first read after each write becomes a miss, and hot keys need extra protection when they repopulate.

Real World

System	How it's used
Redis object caches	Many services pair PostgreSQL or MySQL with Redis in cache-aside mode: GET on read, SET on miss, DEL after writes. It is the standard production and interview example because it is explicit and incremental.
Facebook-style Memcached invalidation	Large web systems often prefer delete-on-write for object caches because deleting stale copies is safer than trying to update every derived key immediately.

Interview

Questions interviewers ask

Walk through the read and write path for cache-aside.
Why is delete-on-write often safer than update-on-write?
What race can still produce stale data after invalidation?

What a strong answer covers

A strong answer covers cache-first reads, DB-first writes, delete-after-commit invalidation, TTL as a backstop, and the stale repopulation race.

Common traps

Treating TTL as the only freshness mechanism.
Ignoring the stale repopulation race on concurrent read/write overlap.

Quiz

In cache-aside, who is responsible for loading data into cache on a miss?

The database engine automatically
The application code
The load balancer
The operating system page cache

Cache-aside is explicit: the application notices the miss, reads the source of truth, and writes the value back to cache.

Why is delete-on-write often safer than update-on-write in cache-aside systems?

It makes the database optional
It avoids network hops entirely
It guarantees zero cache misses after writes
It reduces the chance of writing a stale value back during ordering races

Deleting after the database commit forces the next fill to come from the source of truth. Updating in place can accidentally repopulate cache with an older value if reads and writes overlap.

What is the main purpose of request coalescing for a hot key?

To turn many simultaneous misses into one origin fetch
To increase TTL automatically
To replicate the cache across regions
To make the cache strongly consistent

Coalescing prevents a herd: one request fetches the value while the others wait or reuse the same in-flight result.

A stale repopulation race happens when:

The cache TTL is longer than one minute
The app uses JSON serialization
A reader fetches an old DB value and writes it back after a newer write has already invalidated the cache
The database and cache are in the same region

The invalidation can succeed and the cache can still become stale again if an older read repopulates after the newer write.

Why is cache-aside a common default in interviews?

It is the only pattern Redis supports
It is explicit, incremental, and still correct when the cache is unavailable
It removes the need for invalidation
It guarantees strong consistency on reads

If the cache fails, the app can still read the database. That graceful fallback is a major reason teams start with cache-aside.

Write-Through

Learn the synchronous write path where each successful write updates both cache and database before the caller gets an acknowledgment.