There is a particular shape of disappointment that follows a successful agentic-AI demo. The demo runs once, looks magical, makes the rounds on Twitter, and then dies the moment it has to run a hundred times — or even ten — against the real world it claimed to handle. The failure isn’t a model failure; the failure is structural. Agent systems are cascading — every step’s error feeds the next step’s input — and any per-step error rate above a very small number produces a system-level error rate of “doesn’t work.”
This post is Volume I of two. It explains why the demo-to-production gap exists, the simple math underneath the cascading-error problem, the super-agent bottleneck that mid-2020s teams keep rebuilding because they don’t realize it’s a known anti-pattern, and the verification-first architectural pattern that the research literature is increasingly converging on. The Production Playbook (Volume II) is the build manual that follows.
What it covers
About eight minutes of reading.
§ 1 — The demo gap. Why one good run looks like a working system and isn’t. The selection bias in demo videos. The honesty test (run the demo a hundred times, count the wins).
§ 2 — Cascading errors, with the math. The simplest argument in the field. If each step is 95% reliable, a five-step task is 77% reliable, a ten-step task is 60%, a twenty-step task is 36%. The numbers compound mercilessly.
§ 3 — The super-agent bottleneck. Why piling more responsibility onto a single agent makes the cascade worse, not better. The pattern most 2024–25 teams reach for and the reason it fails.
§ 4 — The evidence for verification. What the latest research (arXiv 2512.08296) shows: the gap between agent demos and agent production is closed almost entirely by an explicit verifier step. Why verification turns out to be the more important engineering problem than reasoning.
§ 5 — What Volume II will build. A short preview. The nine engineering disciplines that close the gap.
Read it
The post lives at its own URL with a warm-paper editorial layout — signal-orange agent accent against teal verification accent. Volume I of two; the Production Playbook follows.
← Back to Autonomy