Realtime: websockets, presence and push
Request–response assumed the client asks; now the server has something to say first.
Everything so far assumed the client asks first. Chat, live dashboards, collaborative cursors and “driver is 2 minutes away” invert it: the server has news. This chapter is the options ladder for pushing, and the architecture that keeps a million open connections from becoming your defining problem.
The options ladder
| Mechanism | How | Right when |
|---|---|---|
| Short polling | ask every n seconds | lag tolerance ≥ n; brutally simple; wasteful at scale |
| Long polling | server holds the request until news or timeout | near-realtime through hostile proxies; the honest fallback |
| Server-sent events | one HTTP response, kept open, streaming events | one-directional feeds (tickers, notifications) — auto-reconnect built in |
| WebSocket | HTTP upgraded to a persistent two-way frame pipe | bidirectional and chatty: chat, games, collaboration |
The professional instinct is to climb only as high as needed — SSE covers a surprising share of “we need websockets”, at a fraction of the operational surface.
What a million connections actually costs
An idle WebSocket is cheap-ish — tens of KB of buffers and an entry in the event loop (Foundations’ epoll, at last cashing its cheque: a thread per connection died right here). A million idle connections is therefore tens of GB and very careful file-descriptor limits: real, but manageable on a modest fleet. The true cost is architectural: connections are state. The socket tier cannot be casually restarted (every deploy disconnects everyone — stagger it), the L7 balancer must support upgrades and long lives, and “which server holds Alice?” becomes a routing question the next paragraph answers.
The two-tier shape, and fan-out
Keep socket servers dumb: they authenticate, hold connections, and subscribe each to the topics it cares about (room:42, user:alice) on a pub/sub backbone (Redis pub/sub, NATS, Kafka by need). Application servers stay stateless and simply publish — “message in room 42” — and every socket server holding a member delivers it. Nobody asks where Alice is connected; the subscription is the routing table. Presence — who is online, who is typing — is deliberately ephemeral state: TTL’d keys refreshed by heartbeats, living in memory, never the database; it is a derived view of connections, and losing it costs a few seconds of green dots.
Reconnection is the real protocol
Mobile networks guarantee disconnects, so design for the gap, not against it: clients reconnect with jittered backoff (the retry rules, verbatim) carrying a resume token — last-seen sequence per topic — and the server replays what they missed from a short buffer… which is the streams chapter’s log, in miniature: per-topic ordering, consumer offsets, catch-up reads. Beyond the buffer’s horizon, fall back to “refetch state via the API”. And when no socket exists at all — app backgrounded, phone in a pocket — mobile push (APNs/FCM) is the delivery of last resort: an unreliable, rate-shaped hint to come online, not a message bus; anything important still lands in the inbox model server-side.