Rate limiting and abuse protection
Your API will be called too much — by bugs, by scrapers, by your own retries. Decide the rules before the flood.
Your API will be called too much. Not maliciously, mostly — a partner’s retry loop without backoff, a mobile release polling every second, your own batch job at midnight — and the failure chapter taught what unbounded intake does to a system. Rate limiting is where you decide the rules before the flood, and communicate them so well-behaved clients can stay well-behaved.
The algorithms, honestly compared
| Algorithm | Mechanic | Character |
|---|---|---|
| Fixed window | counter per minute, reset on the boundary | trivial; permits 2× bursts straddling the reset — fine for rough caps only |
| Sliding window counter | weighted blend of this window and last | smooths the straddle cheaply; the pragmatic middle |
| Token bucket | tokens drip in at rate r up to burst b; requests spend one | the default: steady rate plus honest bursts — matches real traffic |
| Leaky bucket | queue drained at fixed rate | smooths output perfectly; adds queueing delay — shaping, not just limiting |
Token bucket wins most arguments because its two knobs map to the two real questions: sustained rate (r) and forgivable burst (b). Stripe’s post in the reading runs exactly this in production, with a detail worth stealing: alongside the per-user limiter sits a load shedder that reserves capacity for critical traffic when the whole system is hot — rate limiting protects you from one client; shedding protects everyone from everything (the failure chapter, at the front door).
Keys, layers, and the distributed counter
What you key on is policy: per API key (the baseline), per user, per IP (weak alone — NATs share IPs, botnets don’t), per tenant, and layered — a per-endpoint cap inside a global cap, expensive endpoints priced lower. Then the systems question: counters where? A shared Redis gives exact global limits and adds a hop plus a hot key at scale (atomic Lua for check-and-spend); per-node local buckets are free and fuzzy (N nodes ≈ N× the limit worst-case). The grown-up answer is the hybrid: coarse local buckets as the first gate, the shared store for the precise global ceiling — and remember the sketches chapter when “roughly over” is good enough. Precision here is a cost knob, not a virtue.
Saying no like a professional
A rate limit response is API surface (the API chapter’s law: it will be depended on): 429 Too Many Requests, a Retry-After that is honest, and limit/remaining/reset headers so clients can pace before hitting the wall. Well-built clients treat 429 + backoff + jitter as routine; your own SDKs should model it. And distinguish the neighbour from the adversary: limits handle the noisy; abuse (credential stuffing, scraping, carding) gets its own playbook — cost-based limits (login attempts priced steeply), concurrency caps, device/behaviour signals, and the occasional tarpit — because an attacker under your rate limit is still an attacker.
Where it bites
- Announcing limits nowhere, then enforcing them — partners discover policy as an outage.
- One global Redis counter as a single point of failure — the limiter must fail open or degrade local, never take the API down with it.
- Limiting after the expensive work — the gate belongs at the gateway, before the fan-out.
- Uniform pricing: a search-everything endpoint costing the same token as a health check.