This is the seventh post in the diagnosis set, and the one that turns watching into proving. Every earlier post inferred a cause from evidence the fleet handed us. This one pushes on the world and measures what moves. The setup is the running scenario: a charging code (P0AE0) concentrated in cars on firmware ≥ v5.1, a cold-throttle bug diagnosed, and a v5.2 patch proposed. The obvious next step — ship it and watch the curve bend — is exactly the trap this post dismantles.

The problem is that a curve can bend for reasons that have nothing to do with your fix: coincidence, seasonality (this is a cold-charge fault, and spring arrives on its own schedule), regression to the mean, or any of the other changes shipping in the same week. The cure is to stop relying on a single before/after line and build a comparison — a group that lived through the same week without the change. We show how to construct that comparison cleanly when you can randomize, and cleverly when you can’t.

We keep the running scenario from the previous posts — a fleet of 6,000 EVs, one charging code, a firmware cause already diagnosed and now to be proven fixed.

What it covers

Ten sections, about twenty-three minutes, in plain language with the arithmetic shown.

Part A · Why observation isn’t proof

§ 2 — The limits of adjust-and-watch. Confounders, and the unbeatable problem of the common cause you never measured — the ceiling of every observational study.

§ 3 — Draw the causal graph. DAGs, simply: nodes and arrows, the back-door path you must adjust for, and the collider trap you must not adjust for.

Part B · Make it interventional

§ 4 — The staged rollout as a randomized experiment. Assign v5.2 by coin flip and the staged rollout becomes a randomized controlled trial — treatment vs. control, a clean causal effect (the do-operator, in plain words).

§ 5 — Difference-in-differences. When you can’t randomize: compare treated vs. not-yet-updated cohorts before and after, difference out the shared trend, and defend the parallel-trends assumption.

§ 6 — Instrumental variables and natural experiments. When luck assigns the treatment — rollout order, regional scheduling — and you borrow that as-good-as-random variation as an instrument.

Part C · Run it well

§ 7 — Designing the experiment. Sample size and power, a safe ramp (canary → 10% → 50% → 100%), guardrail metrics, and a stopping rule.

§ 8 — Worked example. A 10% randomized canary, treatment vs. control with a confidence interval, and a DiD cross-check against the not-yet-updated cohort — the estimated causal effect of v5.2.

§ 9 — Practical notes. Field rules for proving cause.

§ 10 — References & further reading. Causal inference, econometrics, and online controlled experiments, with pointers.

Read it

Open the monograph →

The monograph lives at its own URL in the warm-paper layout of the series, with a confounder graph, a confounder-vs-collider DAG panel, a randomized treatment/control split, a difference-in-differences two-line plot, an instrumental-variable path diagram, a ramp-with-guardrails figure, a worked confidence-interval box and DiD cross-check table, two algorithms, practical notes, and references.

Where it sits

The diagnosis series, in order:

I · The core method

  1. Root-Cause Analysis Under Uncertain, Heterogeneous Evidence
  2. When Is the Evidence Enough?
  3. Harder Cases in Diagnosis

II · At fleet scale

  1. Fleet-Scale Root-Cause Analysis
  2. Fleet RCA, Made Rigorous

III · Proof, foundations & prevention

  1. Proving Cause with Experiments — (this post)
  2. Diagnosis on the Vehicle
  3. The Data Plumbing Behind Fleet Diagnosis
  4. From Root Cause to No Cause

Full map: The Diagnosis Series index.

← Back to Autonomy