← Learning
All pathwaysTech blogsPapers
Pathway 03 · zero → professional

System Design

From one server to a defensible architecture

The applied track, and the widest: requests, APIs, load balancing, caching, storage, transactions, sharding, streams, derived data, probabilistic structures, geo-indexing, IDs, files, realtime, background jobs, rate limiting, security, resilience, delivery — closing with a worked design and an atlas of the classic problems.

24 chapters~250 min total38 links out0 accounts required
The chapters — one page each
  1. 01Anatomy of a requestEvery system you will ever design is, at bottom, this one picture with more boxes.
  2. 02Designing the APIThe API outlives every implementation behind it; design it like the permanent thing it is.
  3. 03Load balancing, gateways and discoveryThe boxes between the internet and your code deserve to be understood, not inherited.
  4. 04Scaling readsTraffic grows read-first. The first real architecture most systems grow is a cache in front of a database.
  5. 05The edge: CDNs and static deliveryThe fastest request is one that never crosses an ocean; the edge is how you arrange that at scale.
  6. 06Where the data actually livesThe database stops being a black box: two families of storage engine, and the workloads each one disappoints.
  7. 07Data modelling across paradigmsThe engine was half the story; the shape you store is the other half, and it is chosen, not discovered.
  8. 08Getting writes rightReads can be a little stale and nobody dies. Writes are where correctness lives.
  9. 09Sharding and partitioningWrites outgrow one machine; how you split the data decides which queries stay easy.
  10. 10Streams and the logPast a certain scale the question changes from “where is the data?” to “in what order did things happen?”
  11. 11Derived data: search, analytics and event sourcingOne source of truth, many read-optimised shadows — kept honest by the log you just built.
  12. 12Counting at scale: probabilistic structuresExact answers cost memory linear in the data; a whole family of structures trades a tunable error for constant space.
  13. 13Geospatial indexing and proximity“Find drivers near me” breaks every index you have met so far; space needs structures of its own.
  14. 14Identity, time and unique IDsAuto-increment dies with the single database; whatever replaces it must be unique, fast, and ideally sortable.
  15. 15Files, media and blob storageThe database stores who uploaded the video; something else entirely stores the video.
  16. 16Realtime: websockets, presence and pushRequest–response assumed the client asks; now the server has something to say first.
  17. 17Background work: queues, schedulers and cron at scaleNot everything happens in a request; the work that happens later needs the same design care, and usually gets none.
  18. 18Rate limiting and abuse protectionYour API will be called too much — by bugs, by scrapers, by your own retries. Decide the rules before the flood.
  19. 19Identity, authorisation and the security basicsEvery design so far assumed the caller is who they claim; that assumption is a subsystem.
  20. 20Designing for failureThe system works. This stage is about the day it doesn’t, designed in advance.
  21. 21Multi-region and disaster recoveryOne region is a single point of failure with excellent marketing; going beyond it is a design, not a checkbox.
  22. 22Shipping safely: deploys, flags and migrationsMost outages walk in through the front door, wearing a deploy. Ship in a way that assumes so.
  23. 23A worked design, end to endEverything in one sitting: a real problem taken from requirements to a defensible architecture, out loud.
  24. 24The problem atlas: classic designs decomposedThe canonical problems, each reduced to the stages that make it hard — the map from prompt to chapter.