Systems that shipped

Theory earns its keep when it meets a workload. Three famous systems, three different answers to the same question — which guarantee do we sell, and which do we give up? Read them with the vocabulary you now have, and they stop being papers and become priced decisions.

Dynamo — availability as the product

Amazon’s constraint was blunt: the shopping cart must accept writes even during partitions; a refused “add to cart” costs real money, a briefly-forked cart does not. Everything in the design follows. Keys are spread with consistent hashing on a ring — each node owns arcs of key space via many virtual nodes, so adding or removing a machine moves only ~1/N of the data (contrast hash mod N, which reshuffles nearly everything). Membership spreads by gossip; there is no leader anywhere. Writes use the sloppy quorums and hinted handoff you met in Stage 3; divergent replicas are found cheaply in the background by comparing Merkle trees; and when versions genuinely conflict, Dynamo keeps both, ships them to the application with their vector clocks, and lets it merge (carts: union). Every stage of this pathway appears in one system, tuned entirely toward “never refuse a write”.

Spanner — buying back the clock

Google wanted the opposite pole: globally distributed data with real transactions and external consistency — if transaction B starts after A finishes, anywhere on Earth, B’s timestamp is after A’s. Stage 2 said wall clocks cannot give you that. Spanner agrees, then attacks the premise: TrueTime returns not a timestamp but an interval — “now is somewhere in [earliest, latest]” — kept a few milliseconds wide by GPS and atomic clocks in every datacenter. The protocol then simply refuses to lie: a transaction takes a timestamp and waits out the uncertainty (commit-wait) before making its writes visible, so timestamp order provably matches real order. Underneath, each shard is a Paxos group (consensus, Stage 4) and cross-shard transactions run two-phase commit on top of those fault-tolerant groups — 2PC’s blocking flaw neutralised by making every participant itself replicated. The lesson generalises: sometimes the winning move is not a cleverer algorithm but an engineered reduction of the uncertainty everyone else designs around.

MapReduce — making failure boring

The odd one out — computation, not storage — and the origin of a move the data world still runs on: with terabytes in play, move the computation to the data. Express the job as two pure functions: map turns each input record into key–value pairs; the framework shuffles — groups every value by key across the cluster (the expensive, all-to-all step); reduce folds each group to results. Because the functions are pure and inputs immutable, the response to Stage 1’s partial failure is luxuriously dumb: a worker died? Re-run its tasks elsewhere. A worker is merely slow? Run backup tasks — duplicate the stragglers and take whichever finishes, an at-least-once trick that works only because the work is idempotent by construction. Add scheduling for data locality and failure stops being an event at all; it is amortised into the design. Successors (Spark most famously) keep exactly this recovery-by-recomputation idea and fix the economics — intermediate data in memory instead of on disk between every step.

How to read the next one

These three give you a permanent reading protocol for any systems paper or architecture post: what failure model is assumed? where does order come from? which consistency rung is sold, and to whom? what happens during a partition? what got cheaper, and who pays? Answer those in the margin as you read, and you are no longer consuming systems — you are pricing them. The System Design pathway now puts that pricing to work on things you will actually build.