Scaling reads
Traffic grows read-first. The first real architecture most systems grow is a cache in front of a database.
Growth arrives read-first: most systems serve tens or hundreds of reads per write. So the first real architecture almost every product grows is the same one — put the answers people keep asking for somewhere cheaper than computing them again. A cache is Foundations’ ladder used as a tool: copy hot data a few rungs up.
The layers
Browser caches spare you the request entirely (cache headers are the cheapest performance feature you will ever ship). The CDN answers from the edge. An in-memory tier — Redis or memcached — sits in front of the database holding hot keys at sub-millisecond cost. The database caches pages underneath it all. Each layer absorbs some fraction, so the layer below sees only what leaked through. The governing number is the hit rate, and it is worth feeling its leverage: at 99% the database sees 1% of reads; a deploy that flushes the cache raises that to 100% instantly — a 100× traffic spike from a “cache blip”, which is how caches cause the outages they were bought to prevent. Warm-up and gradual rollout are part of cache design, not operational trivia.
The four write policies
| Pattern | How it works | Cost you accept |
|---|---|---|
| Cache-aside | app reads cache; on miss reads DB and fills; writes go to DB and invalidate | the default; a stale window between write and invalidation |
| Read-through | cache itself loads on miss | same behaviour, library-shaped |
| Write-through | writes go to cache + DB synchronously | slower writes; cache always fresh |
| Write-behind | writes hit cache, flushed to DB async | fast writes; you can lose acknowledged data — use with eyes open |
Staleness policy is a per-data-class decision, and this is replication lag from the Distributed Systems pathway wearing a cheaper suit: a TTL where bounded staleness is harmless (a profile, minutes old), explicit invalidation where it bites, versioned keys where you would rather never argue with a stale copy. Write the tolerable staleness down and the policy usually chooses itself.
The failure modes with names
- Stampede (dog-pile): a hot key expires and a thousand requests miss at once, all recomputing. Fixes: per-key locking or request coalescing (one flight recomputes, the rest wait), TTL jitter so keys do not expire in choirs, or probabilistic early refresh of items about to expire.
- Hot key: one celebrity item exceeds what a single cache node serves. Fixes: replicate that key across nodes, or add a tiny in-process cache for the top-N keys — even 1 second of local TTL flattens astonishing load.
- Penetration: requests for keys that do not exist bypass the cache forever. Fix: cache the negative (“no such user”, short TTL).
- Eviction surprise: memory pressure evicts by LRU/LFU policy, not by your intent — capacity-plan the working set, or your “cached” items quietly are not.
Beyond the cache
When one primary still cannot carry residual reads, add read replicas and route queries that tolerate lag to them (the anomalies and patches are exactly the Distributed Systems Stage 3 catalogue — this is where you apply it). And split by job — vertical partitioning: search traffic to a search index, sessions to a key-value store, analytics to a column store (Stage 3 explains why). Sharding the write path is the bigger hammer; it waits for Stage 5.