Not best practices. Not patterns. Physics.
Things on your desk are reachable in a second — but you only have room for a few. The warehouse holds millions of files but takes a day to retrieve from. Software design is largely about deciding what lives on the desk.
Light only travels about thirty centimetres per nanosecond. Distance to the processor matters. Density matters. Cost matters. Stack them together and you get a hierarchy: a sliver of fast memory, then more, slower memory, then more, slower still — all the way out to the warehouse on the other side of the planet.
Each tier is roughly an order of magnitude or more slower than the one above. These are not benchmarks. They are the price list you work from.
Imagine a kitchen. The chef can chop ingredients only so fast. The doorway from the pantry can fit only so many ingredients per minute. When orders pile up, you find out which one was the limit.
Adding a second chef helps if the chef was the bottleneck. It does nothing if the pantry door was. In any moment of any system, one of these two is binding — and the work that helps one rarely helps the other.
One operation is the chef, one is the doorway. They are never on the same scale — and which one is binding decides the entire design.
Coordination is the same problem as a group text deciding on a restaurant. Each round of "how about Italian?" "no, sushi?" takes minutes to converge — and the more people on the thread, the longer it takes.
In distributed systems, each round-trip across the planet takes milliseconds, and getting many machines to agree on anything requires multiple rounds. Two nodes a thousand kilometres apart cannot exchange a round-trip in under ten milliseconds. That is the floor — before any software runs at all.
These are physical floors set by the speed of light in fiber. No software, no cleverness, no money buys lower numbers. Sydney to New York: one hundred and sixty milliseconds, before any code runs.
Imagine a hundred people each wearing a wristwatch. They all set their watches at the same instant. A week later, no two watches agree. None of them is broken — a watch's tick rate depends on tiny physical things: how precisely its mechanism was made, the temperature of the air around it, even how it gets jostled on a wrist. No two watches share exactly the same conditions, so they tick at almost — but not quite — the same speed.
Every server has a watch like that inside it: a tiny crystal that vibrates millions of times a second, and a counter that turns those vibrations into the time. No two crystals vibrate at exactly the same rate, so each server's clock drifts at its own pace — left alone for a day, the drift adds up to roughly a second.
Servers fix this by asking a trusted timekeeper on the internet — "what time is it, really?" — every few minutes, then nudging their own clock back into line. (This service is called .) But the answer is already milliseconds stale by the time it arrives, and the clock keeps drifting between checks. So even two diligent, well-corrected servers still disagree about now by tens of milliseconds.
Even when both servers are doing everything right, they disagree about "now" by tens of milliseconds. That gap is where ordering protocols live.
A hierarchy you can't escape.
A throughput you can't multiply.
An agreement you can't speed up.
A clock you can't trust.
Everything else is a response to one of these.
Caches. Queues. Logs. Hashes. Replicas. Indices. Every primitive in your stack is a workaround for one of the four — a clever way of saying "I cannot move this wall, but I can negotiate with it."
Here are the twenty-five problems engineers face most often. Every one of them is just a different wall.
Pick a tile above. We will dissect it the way the four walls demand — with arithmetic, not adjectives.