Monograph · Human Factors of Agentic AI

The Verification Bottleneck

Does working with frontier AI demand a high IQ? The evidence says something subtler: intelligence sets the entry ticket, but the real constraint is how fast a human can verify machine output, and that constraint is an engineering problem.

01 · The wrong question

Is there an IQ cutoff for working with advanced AI?

A natural worry as models grow more capable: perhaps only people above some cognitive threshold can keep up. The first wave of field evidence pointed the opposite way. In a large controlled study of consultants, AI assistance raised the performance of below-average participants by 43 percent and top performers by only 17 percent, compressing the skill gap[1]. A randomized experiment on professional writing tasks found the same pattern: the largest gains went to the weakest writers[2]. Call-center data showed novices gaining most from an AI assistant while experts gained little[3], and developers using an AI pair programmer completed a standard task 56 percent faster, again with the biggest lift among the less experienced[4].

Economists call this the leveling effect: for routine knowledge work, AI acts as a floor-raiser. If the story ended there, raw intelligence would matter less in the AI era, not more.

02 · The complexity gradient

Leveling below, amplifying above

The story does not end there, because those studies measured tasks near the middle of the complexity range: memos, boilerplate code, scripted support calls. A century of psychometric research shows that the correlation between general cognitive ability and job performance rises with job complexity, reaching its maximum in professional and scientific work[5],[6]. Frontier R&D, deriving a Koopman lifting for a nonlinear plant, judging whether a reinforcement learning formulation is well posed, spotting that a proposed root cause violates conservation of energy, sits at the top of that gradient.

There, the dynamic flips. The model raises the ceiling, and higher-ability people exploit a raised ceiling more, because they can pose better problems, detect subtler errors, and integrate the model's output into a larger structure. Leveling below, amplifying above. Figure 1 lets you walk the gradient yourself.

Figure 1 · Interactive The leveling-to-amplifying transition

Drag the slider from routine work toward frontier R&D. At low complexity the AI-assisted curve (rust) is high and flat: everyone lands near the same output and the gap to the unassisted curve (teal) is largest at the bottom of the skill range. As complexity rises, the assisted curve steepens: the same tool now returns more to those who bring more. Curves are illustrative, shaped to match the qualitative findings of refs. [1] to [6].

03 · Threshold plus multiplier

How much intelligence, and of what kind

Put the two literatures together and the most defensible model of "who works well with frontier AI" is not linear returns to IQ. It is a threshold plus multiplier structure. Below some level of fluid ability[7], a person cannot hold the abstractions needed to verify the model's reasoning about, say, flux linkage estimation; no amount of prompting skill compensates. Above that threshold, the differentiators shift to traits that are correlated with general ability but not identical to it: the speed of absorbing new material, the cost of switching context between parallel threads, and the capacity for creative recombination of ideas.

This matters because working with agentic systems is a high-cycle-rate activity, rapid hypothesis generation, evaluation, and reframing, rather than a single deep thread. An engineer at the 90th percentile of ability with high openness and cheap task switching can outperform one at the 99th percentile who is rigid, because the workflow rewards cycles per hour, not depth per cycle.

Figure 2 · Interactive Threshold-plus-multiplier model of AI-augmented output

Effective output with AI as a function of general ability. Below the threshold (ochre band) the human cannot verify the model's reasoning in the domain, so returns are near zero regardless of tooling. Above it, the slope is set by the multiplier traits: learning velocity, context-switching cost, creative recombination. Raise the multiplier and watch the marked engineer A (90th percentile ability, high flexibility) overtake engineer B (99th percentile, rigid). A conceptual model, not fitted data.

04 · Attention and switching

The supervisory span: switch costs and attention residue

The multiplier traits in Figure 2 have a mechanism, and it lives in the attention literature. At the level of task sets, attention is not a parallel resource; it is a serial bottleneck that must be reconfigured every time you move between threads, and the reconfiguration is not free. Task-switching experiments measure a reliable switch cost, slowed and more error-prone performance immediately after every change of task set, persisting even with full warning and practice[14]. On top of it rides attention residue: when you leave thread A for thread B, part of your attention stays attached to A, degrading judgment on B, and the effect is strongest when A was left unresolved[15].

Both taxes are paid per switch, so they compound with the number of parallel agent threads you supervise. The result is an interior optimum, a supervisory span: the same fan-out limit measured when a single operator controls multiple robots[16]. Below the span, the human idles between reviews. Beyond it, per-thread scrutiny collapses and exception review degenerates into the rubber-stamping of Section 07 one step early. Figure 3 lets you find the span and watch it move.

Figure 3 · Interactive Find your supervisory span

Effectively verified output per hour versus parallel threads supervised. The dashed line is perfect parallelism, a human with no switching costs. The solid curve pays the switch-cost and residue tax on every transition; the ochre marker is the resulting supervisory span. Raise either tax and the span shrinks toward deep single-threading; cut both and it grows. An illustrative model in the spirit of the fan-out literature [16], not fitted data.

The span is not fixed, and the levers for widening it are the ones this monograph keeps arriving at. Fixed schemas cut the per-switch reconfiguration cost, because re-entering a familiar shape is recognition rather than reorientation. Clean closure points cut residue: an agent should hand you a thread at a resolved state, conclusion, action, breaks-if, never mid-thought[15]. And batching beats interleaving: reviewing three finished reports in sequence costs two cheap switches, while three interleaved conversations cost dozens of expensive ones.

05 · The real constraint

Bandwidth is supervision capacity

Grant the threshold is met. The binding constraint on a human working with a frontier model is then not comprehension speed but time-to-verification: how long it takes to decide whether a given output is trustworthy enough to act on. A model can draft ten root-cause hypotheses in the time it takes to read one carefully. The human becomes the slow stage in the pipeline, and every queueing argument says the system throughput is set by its slowest stage.

The bottleneck is not how fast you read. It is how fast you can answer three questions: what is the claim, what is it based on, and where could it be wrong.

Cognitive load theory explains why raw prose is the worst possible interface for this: extraneous load, the effort of finding structure in unstructured text, competes directly with the germane load of actually judging correctness[8]. And working memory holds only a handful of items at once[9], so any report that forces the reader to hold intermediate results while hunting for the conclusion is spending the scarcest resource in the loop.

The mature answer is the one every high-throughput human supervision system converged on, from air traffic control to code review at scale: manage by exception. Machines check everything; humans review violations plus a sample of passes. Figure 4 shows the pipeline.

Figure 4 · Interactive block diagram The exception-first verification pipeline

Tap or hover a block to read its role. Rust path: exceptions. Teal path: sampled audits of "green" output, the guard against rubber-stamping.

06 · Engineering the output

Formatting as a bandwidth technology

Between the verifier layer and the human sits the report itself, and its format is worth engineering with the same seriousness as the verifiers. Five rules do most of the work:

Bottom line up front, in layers. A five-line executive layer stating the conclusion and required action; a twenty-line reasoning skeleton; full detail collapsed below. The reader always consumes layer one, opens layer two only when something smells off, and touches layer three rarely. This is the military BLUF convention[10] fused with progressive disclosure.

Provenance and confidence tags. Every load-bearing claim is marked as derived, computed, retrieved, assumed, or judgment, with a confidence level. The tags tell the reader where to spend attention: skip the high-confidence computed results, audit the assumptions.

Falsification pointers. The report names the conditions under which its own conclusion fails, and the cheapest check for each. The model pre-computes the reader's verification checklist.

Delta-only iteration. On revision, report what changed and why, never the whole state again.

Semantic emphasis and closure. Bold is allocated by decision-relevance, verdicts, deviations, budget-breaking numbers, unverified assumptions, never mechanically; uniform salience is no salience, which is why letter-bolding schemes fail under testing. And every report ends with a NEXT block: candidate moves ranked, one marked recommended with its reason, each alternative carrying a choose-instead-if condition. Ending at a resolved state is what lets the reader release the thread without the attention residue of Section 04[15].

Fixed schemas. Identical section order for every report of a given type, because recognition is dramatically faster than deliberate comprehension[11]. After twenty reports in one shape, the eye jumps straight to anomalies.

Figure 5 · Live demo A layered diagnostic report, as the human sees it
CONCLUSION Pack thermal derate on vehicle 7A traces to a degraded coolant pump, not cell aging. Confidence high. ACTION Pull pump duty-cycle log before approving any warranty cell replacement. Breaks if: · Coolant flow sensor itself has drifted (check: cross-compare against pump speed map) · Derate events correlate with ambient above 38 C only (check: filter events by T_amb) NEXT · RECOMMENDED: commanded pump duty sweep at service; cheapest test that separates the top two hypotheses · Replace flow sensor first; choose instead if its calibration record already shows drift · Monitor 2 more weeks; choose instead if the vehicle is not derating in customer use REASONING · Cell dT/dt under load is 2.1x baseline [computed, high] · Impedance growth is within 4% of fleet median [computed, high], inconsistent with aging as primary cause [derived, high] · Coolant outlet-inlet delta collapsed from 6.2 C to 1.8 C over 30 days [computed, high] · Pump duty command rose 22% for the same flow request [retrieved: fleet telemetry, high] · Flow sensor assumed within calibration [assumed] · Strongest counter-evidence: two derate events at low ambient with normal delta-T; explainable by regen transient [judgment, medium] FULL DETAIL Eliminated candidates: BMS sensor fault (self-test clean), stuck valve (position feedback nominal), cell ISC (no voltage signature). Thermal model residual analysis, raw telemetry windows, and the impedance regression are attached in the run artifact. Next measurement with highest information: commanded pump sweep at service, measured flow vs duty curve against the factory map.

Layer one alone answers what, on what basis, and where it could be wrong, in under thirty seconds, and the closing NEXT block hands over a recommended move with one-glance override conditions, so the thread ends at a resolved state. Toggle the tags: with highlighting on, the single unverified assumption announces itself. The example is synthetic.

07 · The failure mode

Rubber-stamping, and the discipline against it

Exception-first review has one well-documented failure mode. Human vigilance over a stream of mostly-fine signals decays measurably within the first half hour of watch[12], and trust in automation drifts toward complacency precisely when the automation is usually right[13]. A reviewer who has seen four hundred green reports stops seeing report four hundred and one.

The countermeasure is structural, not motivational: the workflow itself must randomly select a fraction of passing outputs for deep audit, and the report format should end by naming the audit target so the discipline is built into the artifact rather than left to willpower. Try it yourself: one of the eight "passing" checks below contains a real anomaly.

Figure 6 · Exercise Find the anomaly in the green stream
Click the row that should not have passed.

All eight rows render with a green status dot, exactly as a passing verifier summary would. One violates a physical or dimensional constraint. This is what your attention has to do at row four hundred and one, which is why sampling must be automated rather than willed.

08 · Synthesis

What to conclude

Three claims survive the evidence. First, for routine work, AI levels: cognitive ability matters less than it did. Second, for frontier technical work, AI amplifies: high general ability is an entry condition, and past the threshold, returns shift to learning velocity, cognitive flexibility, and creative recombination, which are exactly the traits AI extends rather than replaces. Third, the operative constraint day to day is verification bandwidth, an attention budget taxed by every context switch, and unlike IQ, verification bandwidth is engineerable: verifier agents, exception queues, layered reports, provenance tags, fixed schemas, and automated audit sampling can multiply how much frontier-model output one careful human can responsibly supervise.

The practical implication is almost optimistic. You cannot add thirty points of fluid intelligence, but you can restructure every report that reaches your eyes so that the first thirty seconds do the work of thirty minutes. The bottleneck is real, and it is yours to widen[a].


Footnotes
[a] The output-formatting rules of Sections 04 and 06, packaged as a reusable agent skill (v2)

The following skill file makes an AI assistant apply the layered format, provenance tags, falsification pointers, delta-only iteration, exception-first reporting, and audit-sampling reminder to every substantive output automatically. Drop it into a skills directory, or adapt the body as custom instructions for the agent framework of your choice.

Download the full skill (v2, with the code-output, review-map, and session-ledger rules the excerpt below omits): human-bandwidth-output-skill.zip

---
name: human-bandwidth-output
description: Format every substantive response so a human engineer can
verify it in seconds, not minutes, and always close with a ranked NEXT
block naming the recommended move. Use for analysis, root cause
reports, design and code reviews, derivations, simulation results,
batch or agentic run reports, and any multi-paragraph technical output.
---

Purpose
The reader is a domain expert reviewing complex engineering output at
high pace. The bottleneck is time-to-verification, not reading speed.
Every output must let the reader answer three questions within the
first 30 seconds: (1) What is the claim? (2) What is it based on?
(3) Where could it be wrong?

Layered structure
LAYER 1, BLUF, max 5 lines, each under 80 characters so it never wraps
in a narrow chat panel. Conclusion with confidence first, no preamble,
one ACTION line.
LAYER 2, reasoning skeleton, max 20 lines, one tagged claim per line,
including the strongest counter-evidence encountered.
LAYER 3, full detail under its own heading, skippable, never in
contradiction with the layers above.

Review map (any output over ~40 lines)
After Layer 1, a routing block: what to check, in what order, with
time estimates, ordered by risk times leverage. It is a contract:
nothing outside the mapped items can change the conclusion.

Tagging
[derived] [computed] [retrieved] [assumed] [judgment], each with
confidence high, medium, or low, inline. Never [computed] unless
execution actually occurred in this session.

Falsification pointers
A "Breaks if:" block, 2 to 4 failure conditions, each with the
cheapest concrete check (a signal name, a file:line, a command).

Chunking and anchoring
Max 4 items per flat list; beyond that, labeled groups of at most 4.
Every number carries units and a comparison anchor ("2.3 ms, vs 10 ms
budget"). New designs described as "same as [known thing] except
[differences]", exceptions first. Aligned tables for 3+ compared
quantities. One name per concept for the whole session; a symbol
table before any new notation; an inline dimensional check on every
derived equation.

Code output (IDE context)
Diff-first: changed hunks only, ~3 lines of context, each hunk with a
one-line intent header stating why. Hunks ranked logic over interface
over rename over formatting, reviewed in that order. Blast radius
stated with clickable file:line anchors. Existing code quoted
verbatim with file:line, never paraphrased. One line on how the
change was, or was not, verified.

Session ledger
Running Assumptions A1, A2, ... and Decisions D1, D2, ... with stable
IDs, referenced instead of restated, every revision flagged loudly
with its downstream consequences.

Delta-only iteration
Open with "CHANGED SINCE LAST:" listing only what changed, why, and
which conclusions or ledger entries are affected.

Expectation versus actual
State expected, observed, and elaborate only deviations. Matching
results get one line; deviations get the analysis.

Exception-first reporting (batch and agentic runs)
Violations first with severity and location, then one-line pass
counts, then a random sample of 2 to 3 passing cases in auditable
detail, then an explicit statement of what was NOT checked.

Emphasis discipline
Bold only decision-relevant tokens: verdicts, deviations,
budget-breaking numbers, changed conclusions, [assumed] tags. Never
mechanical letter-bolding schemes; when everything is emphasized,
nothing is.

Closure: the NEXT block (mandatory)
Every response ends at a resolved state, never mid-thought, so the
reader can release the thread without attention residue. The final
block, NEXT: 2 to 4 candidate moves ordered by expected value,
exactly one marked RECOMMENDED with a one-line reason tied to
information gain, risk reduction, or cost, and each alternative
carrying a "choose instead if [condition]" clause. If only one
sensible move exists, one line, no filler alternatives. For
exception-first reports, the audit suggestion ("deep-check item #N",
N random among the passes, category rotated) is the true last line.
References
  1. Dell'Acqua, F., McFowland, E., Mollick, E., et al. (2023). Navigating the jagged technological frontier: field experimental evidence of the effects of AI on knowledge worker productivity and quality. Harvard Business School Working Paper 24-013.
  2. Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187-192.
  3. Brynjolfsson, E., Li, D., & Raymond, L. (2025). Generative AI at work. Quarterly Journal of Economics, 140(2), 889-942.
  4. Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). The impact of AI on developer productivity: evidence from GitHub Copilot. arXiv:2302.06590.
  5. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262-274.
  6. Gottfredson, L. S. (1997). Why g matters: the complexity of everyday life. Intelligence, 24(1), 79-132.
  7. Cattell, R. B. (1971). Abilities: Their Structure, Growth, and Action. Houghton Mifflin.
  8. Sweller, J. (1988). Cognitive load during problem solving: effects on learning. Cognitive Science, 12(2), 257-285.
  9. Cowan, N. (2001). The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87-114.
  10. U.S. Army (1985 onward). Effective military writing doctrine, "bottom line up front", codified in AR 25-50, Preparing and Managing Correspondence.
  11. Klein, G. (1998). Sources of Power: How People Make Decisions. MIT Press. (Recognition-primed decision model.)
  12. Mackworth, N. H. (1948). The breakdown of vigilance during prolonged visual search. Quarterly Journal of Experimental Psychology, 1(1), 6-21.
  13. Parasuraman, R., & Riley, V. (1997). Humans and automation: use, misuse, disuse, abuse. Human Factors, 39(2), 230-253.
  14. Monsell, S. (2003). Task switching. Trends in Cognitive Sciences, 7(3), 134-140.
  15. Leroy, S. (2009). Why is it so hard to do my work? The challenge of attention residue when switching between work tasks. Organizational Behavior and Human Decision Processes, 109(2), 168-181.
  16. Olsen, D. R., & Wood, S. B. (2004). Fan-out: measuring human control of multiple robots. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2004), 231-238.