Diagnostics · Prognostics · A Worked Case
A fleet of batteries is aging too fast. The statistics point one way, the physics another. This is how you tell a true cause from a convincing impostor.
We follow a single anomaly — a cohort of electric vehicles losing battery health faster than the model allows — from first alarm to confirmed root cause and forward into a prognosis. Along the way we meet the central hazard of diagnostics: a feature that correlates beautifully with failure but causes nothing. The tool that rescues us is abduction — inference to the best explanation — disciplined by partial correlation and, finally, by physics. All figures are computed from a simulated 240-vehicle fleet with a known ground truth, so we can grade our own detective work.
It starts as a line on a dashboard that bends the wrong way.
You monitor a fleet of 240 vehicles. The battery state-of-health (SOH) — estimated onboard from capacity fade and internal-resistance rise — is supposed to decline gently, a few percent a year. Instead, a sizeable slice of the fleet is shedding health at roughly double the baseline rate. No crashes, no thermal events, no diagnostic trouble codes. Just a population quietly dividing into two: the ordinary, and the alarming.
The first job is not to explain it. The first job is to see its shape. When we plot SOH against age and color by the two emerging groups, the fleet is unmistakably bimodal — two trajectories, not one cloud.
Two trajectories means two regimes — something is sorting these vehicles into camps. The investigation is the search for the sorting variable, and for the mechanism behind it.
Before chasing causes, fix the vocabulary, because the two halves of the job have different standards of proof.
Diagnosis is a snapshot: given the evidence now, what is the fault and what mechanism produced it? Prognosis is the projection: given that mechanism and the operating profile ahead, how much useful life remains? The hinge between them is the mechanism. Diagnose only a correlation and your prognosis is a guess with error bars wide enough to drive through. Diagnose the mechanism and the same data becomes a degradation model you can extrapolate, intervene on, and price.
Good diagnostics begins by being generous, then ruthless. We enumerate every plausible cause first, deliberately spanning the failure-mode families so we don't quietly assume the answer:
| Family | Hypothesis for this fleet |
|---|---|
| Thermal | Hot-climate operation accelerates calendar aging (Arrhenius). |
| Usage / behavior | Frequent DC fast-charging stresses cells. |
| Usage / storage | Long dwell at high state-of-charge drives calendar fade. |
| Mechanical / wear | Cumulative cycling (depth × count) wears the electrodes. |
| Electrical / control | A balancing fault lets cells drift apart, over-stressing the weak ones. |
| Software / firmware | A specific build mismanages the battery — the upstream suspect. |
Each is a story about a cause. None is yet measurable. The next step is the one beginners skip: translating each story into a number we can actually pull off the bus.
A hypothesis you cannot measure is a hypothesis you cannot reject. So we map each candidate to a feature already present in the telemetry — CAN logs, BMS snapshots, charge records:
| Hypothesis | Measurable feature | Source |
|---|---|---|
| Thermal aging | Mean ambient temperature (°C) | BMS / climate log |
| Fast-charging stress | DC fast-charge events / month | Charge records |
| High-SOC storage | % time parked above 80% SOC | BMS state log |
| Cycling wear | Equivalent full cycles | Coulomb counter |
| Balancing fault | Post-charge cell-voltage spread, ΔV (mV) | Per-cell BMS telemetry |
| Firmware | Build version (categorical) | ECU manifest |
Now the outcome — annualized SOH fade rate (%/yr) — and six candidate features sit in one table, 240 rows deep. The detective work can begin.
The reflex move — and the right place to start — is to correlate each feature against the fade rate and rank them.
# Step 1 — rank candidates by raw association with fade rate from scipy import stats for name, x in features.items(): r, p = stats.pearsonr(x, degradation_rate) print(f"{name:24s} r={r:+.3f} p={p:.1e}")
Two features stand out: cell-imbalance ΔV and fast-charge frequency. The fast-charging result is seductive: it has a clean engineering narrative (fast charging is "hard on batteries," everyone knows that), a strong coefficient, and a vanishingly small p-value. Plot it and it looks like a smoking gun.
Here is the discipline that separates a diagnosis from a coincidence. Before accepting fast-charging, we ask the four questions every candidate root cause must survive:
| Test | Fast-charging |
|---|---|
| 1. Correlated? | Yes — strongly (r ≈ ). |
| 2. Statistically significant? | Yes — p ≪ 0.001. |
| 3. Explains real variance? | Yes — alone it accounts for a large share. |
| 4. Mechanistically plausible? | This is where it stumbles. |
Three of four pass. The fourth is the one that matters, and it is uncomfortable. When we look at the actual charge sessions, the DC fast-charges in this fleet are within rated C-rate, the pack temperatures during them stay inside the thermal window, and the cell chemistry is qualified for exactly this duty. There is no mechanism by which charging that the cell is specified to tolerate should, by itself, double the fade rate. The number is loud; the physics says nothing.
We are not deducing (we have no law that forces a conclusion) and we are not blindly guessing. We are doing what a diagnostician actually does: surveying the candidate explanations and asking which one best accounts for the whole pattern — the strong fast-charge correlation, the absent mechanism, and the equally strong ΔV signal. That is abduction: inference to the best explanation.
The best explanation for "strong correlation, no mechanism" is almost always a confounder: a hidden third variable that drives both the suspect feature and the outcome. So we form a new, sharper hypothesis:
Fast-charging does not cause the fade. Both fast-charging and the fade are downstream of something else — and the firmware coloring in Figure 3 is the hint. Suppose a firmware build mismanages cell balancing and ships an aggressive charge profile. Then its cars both fast-charge more and degrade faster, with no causal arrow between those two at all.
If ΔV (the balancing signal) is the real driver, then among cars with the same ΔV, fast-charging should carry no extra information about fade. We compute the partial correlation — the correlation of each feature with fade after regressing out ΔV:
# Partial correlation: corr(feature, fade) with ΔV held constant def partial_corr(x, y, z): rx = x - np.polyval(np.polyfit(z, x, 1), z) # strip z from x ry = y - np.polyval(np.polyfit(z, y, 1), z) # strip z from y return stats.pearsonr(rx, ry) # what's left partial_corr(fast_charge, fade, z=cell_imbalance_dV)
The mirror-image test confirms it: hold fast-charging constant and ΔV barely moves (r = ). The dependence runs one way. A multiple regression tells the same story in coefficients — the moment ΔV enters the model, the fast-charge coefficient falls to near zero while the model's explanatory power jumps:
| Model | Fast-charge coef. | ΔV coef. | R² |
|---|---|---|---|
| fade ~ fast-charge + ambient | — | ||
| fade ~ fast-charge + ambient + ΔV |
Abduction has done its work: it took an anomaly the raw statistics couldn't resolve and proposed a structure — a confounder — that the partial-correlation test then confirmed. But we are not finished. A surviving statistical signal is still only a strong clue. It must now pass the test fast-charging failed: physics.
ΔV survived every statistical test. Now we demand a mechanism — and unlike fast-charging, ΔV has one ready. The question is concrete: why would a few millivolts of post-charge cell spread accelerate fade?
A battery pack charges as a series string under one current. The BMS terminates charge on pack voltage. If balancing is healthy, all cells sit at nearly the same voltage and rise together, each topping out safely below the over-voltage stress line. But if balancing fails, the cells drift apart — and the weakest, highest-voltage cell is driven past its limit while the pack average still looks fine. We pull the per-cell traces from a balanced pack and an imbalanced one during a single charge:
Now the four tests all pass for ΔV at once: strong correlation, significant, dominant share of variance, and a clean, observable physical mechanism. This is what a root cause looks like when you've actually found it — the numbers and the physics stop arguing and start agreeing.
ΔV is the mechanism, but a mechanism has an origin. Cells don't drift apart on their own — something stopped balancing them. We stratify the fleet by firmware build, the last suspect on our original list:
Inspecting the build confirms it: a timing bug in the cell-balancing controller — it only equalizes during a narrow state window and misses the windows where the weak cell is drifting. The same build also carried an aggressive DC-charge default, which is why its cars fast-charged more — and why fast-charging looked guilty in Figure 3. The phantom correlation finally has an honest explanation.
The full causal chain, recovered end to end:
firmware v3.2.1 bug → balancing fails → cell ΔV grows
→ weak cell over-volts each charge → accelerated SOH fade
(same build's aggressive charge default → more fast-charging = the red herring)
The case rests on recognizing that "correlated with failure" resolves into three genuinely different roles — and that raw correlation cannot tell them apart:
Strong in the raw ranking, undiminished under control, and backed by an observable physical pathway. Accept — and act on it.
Loud in the raw ranking, gone the moment ΔV is held constant, and never had a mechanism. A confounded proxy for the firmware build. Reject — throttling charge rates would have fixed nothing.
Nearly invisible in the raw ranking, yet a real secondary cause revealed once ΔV is removed — a genuine Arrhenius effect, masked by the dominant driver. Keep it in the prognostic model; it is not the headline.
Note the symmetry of the two failures of naïve correlation: one feature looked important and wasn't; another looked unimportant and was. The same instrument — partial correlation, then physics — corrected both.
Stripped of the battery specifics, here is the move that did the real work — repeatable on any diagnostic problem:
The fleet split into two fade regimes. An anomaly worth explaining is a pattern your current model didn't predict.
Span the failure-mode families before testing any one, so the answer isn't smuggled in as an assumption.
A hypothesis you can't measure is one you can't reject. Map suspicion to telemetry.
Raw correlation is a flashlight, not a verdict. It shows you where to look, including at impostors.
Strong correlation + no physics = a clue about a hidden variable, never a conclusion. This is the trigger to reason abductively.
"Confounder" is the usual best explanation for loud-stats-no-mechanism. Name the hidden driver and predict what holding it constant should do.
True causes survive controls; proxies collapse; masked contributors emerge. The data sorts itself once you ask the conditional question.
The surviving candidate must show its mechanism in the raw signal — here, the weak cell crossing the over-voltage line on every charge.
Because we found the mechanism and not merely a correlate, the diagnosis converts directly into a prognosis. We know ΔV drives fade and we know the firmware fix collapses ΔV back to the healthy range. So for an affected vehicle — say, one currently at 91% SOH and 14 months old — we can project two futures and put a number on the intervention:
And it generalizes: any vehicle on build v3.2.1 carries the same latent risk, even those not yet visibly degrading. The diagnosis becomes a fleet-wide screen — flag every affected VIN, prioritize by current ΔV, push the OTA, and verify recovery in the post-patch telemetry. Diagnosis told us why; prognosis told us who's next and how much time we have.
A root cause nobody can follow is a root cause nobody will trust. The investigation above should be logged not as a conclusion but as a decision path — the artifact that lets a reviewer, a regulator, or a downstream automated agent reconstruct exactly how you got there:
# Decision record — one node per inference, machine- and human-readable { "observation": "bimodal SOH fade; fast cohort ~2x baseline", "hypotheses": ["thermal","fast_charge","high_soc","cycling","balancing","firmware"], "screen": {"method":"pearson", "top":["dV","fast_charge"]}, "challenge": {"feature":"fast_charge", "mechanism":null, "flag":"strong corr, no physics -> suspect confounder"}, "abduction": "both fast_charge and fade are downstream of firmware build", "control": {"partial_r(fast_charge|dV)":-0.09, # collapsed "partial_r(dV|fast_charge)":0.82}, # survived "physics": "weak cell crosses 4.25V on every charge (per-cell trace)", "root_cause": "firmware v3.2.1 balancing timing bug", "action": {"fix":"OTA balancing patch", "screen":"all v3.2.1 VINs"}, "confidence": "high — stats + mechanism + categorical stratification agree" }
Logged this way, every link in the chain carries its own evidence and its own rejected alternatives. That is what makes the diagnosis defensible, the prognosis auditable, and the whole pipeline safe to hand to an autonomous diagnostic agent — which, after all, must reason exactly like this, just faster.
Part II
The case is closed. What follows is the theory underneath every move we made — the concepts that turn a lucky guess into a repeatable method.
They sit on the same data but answer different questions and carry different burdens of proof.
Diagnosis is inverse inference: from observed effects (symptoms, signals, codes) back to the hidden state that produced them. Its standard of proof is mechanistic identification — you must name the fault and the pathway. Prognosis is forward inference: from the identified state and the expected operating profile, project the future trajectory and the time to a threshold. Its standard of proof is predictive calibration — your forecasts must match what actually happens.
The mechanism is the bridge. Formally, if degradation follows some rate law
d(SOH)/dt = f(state, stressors; θ)
then diagnosis is the act of identifying f and its parameters θ, and prognosis is integrating that law forward to find when SOH crosses end-of-life. Diagnose only a correlation and you have no f to integrate — only a line you are extrapolating on faith. Diagnose the mechanism and θ and the same telemetry becomes a calibrated forecast you can act on. This is exactly why, in the case study, finding the balancing mechanism (not just the fast-charge correlation) is what made the remaining-life projection trustworthy.
Hypothesis generation fails most often by omission — the true cause was never on the list. The defense is to enumerate by failure-mode family, because families are how physical systems actually break, and spanning them guarantees coverage:
| Family | Generating mechanism | Signature in data |
|---|---|---|
| Mechanical / wear | Fatigue, friction, cyclic loading | Monotonic drift with usage/cycles |
| Thermal | Arrhenius-accelerated chemistry | Correlates with temperature exposure |
| Electrical / control | Sensor drift, actuator fault, control-loop error | Step changes, regime shifts |
| Software / firmware | Logic defect in a specific build | Categorical split by version |
| Manufacturing | Material or assembly variance | Batch / lot clustering |
| Usage / environment | Abuse, duty cycle, contamination | Correlates with operating profile |
The "signature" column is the practical payoff: each family leaves a different fingerprint, which tells you what to plot. A categorical split (as in our firmware case) looks nothing like a wear process's smooth drift — and recognizing that difference is half the diagnosis.
A candidate is promoted from "associated" to "causal" only by clearing four gates in order. The first three are statistical; the fourth is physical, and it is the one that catches impostors.
| Gate | Question | Tool | What failure looks like |
|---|---|---|---|
| 1 · Association | Do they move together? | Correlation, mutual information | Flat scatter |
| 2 · Significance | Could it be chance? | p-value, confidence interval | Wide CI, large p |
| 3 · Materiality | Does it explain enough? | Effect size, R², variance share | Significant but tiny effect |
| 4 · Mechanism | Is there a physical pathway? | First-principles model, bench test | No story, or story contradicts data |
Fast-charging cleared gates 1–3 and failed gate 4 — that is the exact profile of a confounded variable, and why gate 4 is non-negotiable. A fifth, meta-gate sits above all of these: robustness under control. A true cause keeps its association when you hold other variables constant; a proxy does not. That meta-gate is the subject of §F5.
Diagnostics uses all three, but it is powered by abduction. The distinction is worth making precise, because each has a different job and a different fallibility.
Given a rule and a case, the conclusion is guaranteed. Rule: a balancing failure produces cell drift. Case: this car's balancer is dead. Therefore: this car will show cell drift. Truth-preserving, but it only works once you already have the rule — deduction tests hypotheses, it does not generate them.
Observe many buggy-build cars all showing drift, and generalize: "this build causes drift." Induction builds the rules that deduction later uses. It is probabilistic — the next case could break the pattern — and it is how the statistical screens in the case study earn their keep.
This is the diagnostic engine. We observe an effect (cell drift and accelerated fade) and ask: of all the explanations that could produce this, which one accounts for it best? Abduction is generative — it proposes hypotheses — and it is openly fallible: the best current explanation may be overturned by new evidence. Its selection criteria are explanatory: prefer the hypothesis that covers the most observations, invokes the fewest unsupported assumptions, and coheres with known mechanism.
In practice the three chain into a loop, and naming the loop is what makes the method teachable:
abduce a hypothesis → deduce its testable consequences
→ test inductively against data → update ↻
The whole case study is one turn of this loop: we abduced "confounder," deduced "then fast-charge should lose its signal once ΔV is held constant," and confirmed it inductively in the partial-correlation test.
A confounder is a variable Z that causes both the suspect feature X and the outcome Y. It manufactures an association between X and Y even when no arrow runs from X to Y. Our case is the textbook shape:
firmware (Z) → fast-charge (X)
firmware (Z) → ΔV → fade (Y)
∴ X and Y correlate, with no X → Y
Partial correlation dissolves the illusion. To get the association between X and Y that is not explained by Z, regress Z out of each and correlate what remains:
rXY·Z = corr( X − X̂(Z), Y − Ŷ(Z) )
If every bit of the X–Y association ran through Z, both residuals are now independent and the partial correlation falls to zero — which is precisely what happened to fast-charging (r: 0.74 → −0.09). If X genuinely drives Y, the association survives, as ΔV did.
Ambient temperature did the opposite: weak raw correlation, strong partial. This is suppression (or masking). A dominant driver — ΔV — carried so much of the variance that ambient's genuine but modest effect was buried in the marginal correlation. Removing the dominant variable unmasks the smaller true one. The lesson cuts both ways: raw correlation under-states real causes as readily as it over-states fake ones.
Rejection is not a dead end — it is information. The question is which surviving candidate to test next, and the answer is an economics problem: maximize expected information per unit of effort. Three factors set the order:
Prior probability. What is the base rate of this failure family in your fleet? A known-common mode is worth testing before an exotic one, all else equal.
Coverage. If true, how much of the observed pattern would this hypothesis explain? Prefer candidates that, if confirmed, close the most of the case at once.
Testability. Can you test it with data already in hand, or does it require a bench experiment that takes weeks? Cheap, fast tests come first even at slightly lower prior.
Each rejection then reshapes the field. Viewed through Bayes, observing "test failed" reallocates posterior probability mass across the remaining hypotheses — the eliminated candidate's prior flows to those still standing, sharpening the next choice. Diagnosis is therefore not a checklist run top to bottom but an adaptive search whose ordering updates after every result. This is what we did implicitly when fast-charging's mechanism failure pushed us straight to the confounder hypothesis rather than to the next item on the original list.
Once the rate law f is identified, prognosis is integration. For our fleet the diagnosed mechanism makes ΔV the dominant stressor, so a usable model is
fade_rate ≈ β₀ + β₁·ΔV + β₂·(T − 20)⁺ + …
and remaining useful life is the time until the integrated SOH reaches the end-of-life threshold:
RUL = ( SOH_now − SOH_eol ) / fade_rate
The mechanism does three things no curve-fit can. It tells you which covariates belong in f (ΔV and temperature, not fast-charge). It tells you how an intervention changes the forecast — patch the balancer, ΔV collapses, β₁·ΔV shrinks, the curve bends up. And it lets you screen the unaffected — any car on the buggy build carries the latent risk before it ever shows symptoms. A purely statistical RUL fit can extrapolate the past; only a mechanistic one can answer the counterfactual "what happens if we act," which is the only question a fleet operator actually cares about.
Part III
The same investigation, drawn as a machine: the pipeline it lives in, the decision flow it follows, and the algorithms that execute it.
Every health-monitoring system is the same five-stage pipe: raw signals in, a decision out, and a loop that never stops. Diagnosis and prognosis are two distinct engines inside it, with the diagnostic output feeding the prognostic input.
The arrow between the two engines is the load-bearing one. A prognosis built on a misdiagnosis inherits the error and amplifies it over the projection horizon. That is the systems-level reason the case study spent so long separating a true cause from a confounder before projecting anything.
Strip the battery case to its wiring and it becomes a causal diagram — the "answer key" the analysis was trying to recover. Reading it explains, at a glance, why fast-charging fooled the raw statistics.
This is what §F5 means in pictures: condition on the firmware build (or on ΔV, which carries its effect), and the fast-charge → fade association evaporates because the path that created it is blocked.
Here is the whole method as an operational flowchart — the path any case follows, with the decision diamonds that route it. Note the two ways out of "no": a feature with no mechanism triggers the confounder branch rather than an immediate reject.
The flowchart compiles into three cooperating algorithms. The first drives the search, the second is the confounder test it calls at the critical branch, and the third converts the surviving diagnosis into a forecast.
Now we open the two middle blocks of Diagram 1 and run the example through each in turn. They consume different inputs, run different math, and emit different outputs — but they are joined at the mechanism.
Take VIN on build v3.2.1, currently 91% SOH at 14 months, measured post-charge ΔV ≈ 55 mV:
Detection: fade rate sits well above the fleet baseline band → flag.
Isolation: the anomaly localizes to the pack, and within it to cell-voltage spread, not temperature or cycling.
Attribution: high ΔV + firmware = v3.2.1 + the confirmed over-voltage trace ⇒ root cause is the balancing-controller timing bug. Output: {fault: balancing_bug, mechanism: weak-cell over-voltage, confidence: high}.
The diagnosis hands the prognostic engine the mechanism, which fixes the rate law's covariates (ΔV dominant, ambient secondary). Using the fleet-fitted rates:
| Scenario | Fade rate | RUL to 80% SOH |
|---|---|---|
| No intervention (ΔV ≈ 55 mV) | %/yr | (91 − 80) / rate ≈ mo |
| After OTA fix (ΔV → ≈ 9 mV) | %/yr | ≈ mo |
The intervention roughly the remaining life — a number the operator can weigh against the cost of the OTA campaign. This is the payoff of the whole method: because the diagnostic engine identified a mechanism and not a correlate, the prognostic engine can answer the counterfactual "what happens if we act." Had the diagnosis blamed fast-charging, Algorithm 3 would have applied a fix to s that changes nothing in f — and the RUL line would not have moved at all.
The flowchart in §S3 is the full investigative process. Once the method has run and the causes are known, the recurring per-vehicle decision compresses into a short decision tree of feature thresholds — walk it from the top and any vehicle lands in a fault class in three questions. The split points come straight from the data: the buggy cohort sits near 53 mV ΔV and the healthy near 9 mV, so a 25 mV cut separates them cleanly.
This is the diagnostic engine of Diagram 1 distilled into production form: cheap to evaluate on every vehicle, and every leaf names both a fault and an action. Note the ordering encodes the analysis — ΔV first because it is the dominant, mechanism-backed signal; firmware second to confirm the origin; ambient last as the secondary cause.
Figure 7 drew the average future. Real prognosis is probabilistic: the fade rate varies across vehicles (parameter uncertainty) and wobbles month to month (process noise), so the honest output is a distribution over failure times, not a single line. We make Algorithm 3 stochastic and run a Monte Carlo — 4,000 future SOH trajectories for the affected vehicle, each with its own sampled fade rate, advanced month by month until it crosses the 80% end-of-life threshold.
Plotting the 10th–90th percentile band shows where the vehicle is likely to be over time, not just the median guess:
Integrating the failure times into a cumulative curve answers the operator's actual question — what is the chance this vehicle has failed by month T?
Together these three views are the prognostic engine's full answer: not "it will last 20 months," but a calibrated forecast with its uncertainty, a failure-probability curve to act on, and a counterfactual showing the fix moves the median from to months. Because the underlying diagnosis was mechanistic, every one of these curves responds correctly when we simulate the intervention — the payoff promised back in §F1.
Appendix
Everything in this monograph is reproducible. The complete script and the generated dataset are provided alongside; the essential listings are reproduced here.
The dataset is synthetic by design: to grade a diagnostic method you must know the true answer in advance. We simulate 240 vehicles, planting one real causal chain and one deliberate confounder, then let the analysis try to recover the truth from the data alone. The generated table (fleet.csv) has one row per vehicle with these columns:
| Column | Meaning | Role |
|---|---|---|
| firmware_v321 | 1 if on the buggy build | Root upstream cause |
| cell_imbalance_mV | Post-charge cell ΔV | Mechanism |
| fast_charge_per_mo | DC fast-charge events / month | Confounded proxy |
| ambient_C | Mean ambient temperature | Modest cause |
| cycle_count | Equivalent full cycles | Minor cause |
| soc_dwell_pct | % time parked above 80% SOC | Null (decoy) |
| age_months | Vehicle age | Minor cause |
| fade_rate_pct_yr | Annualized SOH fade rate | Outcome (target) |
Crucially, fast_charge_per_mo enters the fade equation with zero coefficient — any correlation it shows is an artifact of its shared parent, the firmware build. That single design choice is what makes the dataset a fair test of whether the method can resist a convincing impostor.
Each block below is one edge of the causal graph. Read top to bottom, it is the graph itself, written in code:
# === SECTION: data-generation (the mockup data source) =========== # Each block below is one edge of the ground-truth causal graph. # Root upstream cause: ~42% of the fleet runs the buggy build v3.2.1. firmware_buggy = (rng.random(N) < 0.42).astype(int) # Confounding link: the buggy build also shipped an aggressive DC # fast-charge default, so those cars fast-charge more often. # (This is why fast-charging will LOOK guilty but cause nothing.) fast_charge_pm = np.where(firmware_buggy == 1, rng.normal(15, 3.0, N), rng.normal(6, 2.5, N)).clip(0) # Ambient temperature: set by the owner's climate, independent of # firmware. ~40% of owners live in a warm region. region_warm = (rng.random(N) < 0.40).astype(int) ambient_C = np.where(region_warm == 1, rng.normal(31, 3.0, N), rng.normal(18, 4.0, N)) # Nuisance / secondary features. age_months = rng.uniform(6, 36, N) cycle_count = age_months * rng.normal(38, 6, N) # cycles grow with age soc_dwell = rng.uniform(5, 60, N) # % time parked > 80% SOC # THE bridging variable: post-charge cell-voltage spread dV (mV). # Driven almost entirely by the firmware bug; tiny aging term. cell_imbalance_mV = ( np.where(firmware_buggy == 1, rng.normal(52, 8, N), rng.normal(6, 2.5, N)) + 0.004 * cycle_count + rng.normal(0, 2.5, N) ).clip(2) # OUTCOME: annualized SOH fade rate (%/yr). # TRUE drivers = dV (dominant) + ambient (modest) + cycling (small). # NOTE: fast_charge_pm has NO coefficient here -> any correlation it # shows with fade is pure confounding through the firmware build. degr_rate = ( 1.8 + 0.075 * cell_imbalance_mV + 0.075 * np.maximum(ambient_C - 20, 0) + 0.0010 * cycle_count + rng.normal(0, 0.45, N) ).clip(0.5) # Convenience: SOH after 24 months under the current fade rate. soh_24mo = (100 - degr_rate * 2.0).clip(60) # Assemble the table and write the mockup data source to CSV. fields = ["vin", "firmware_v321", "fast_charge_per_mo", "ambient_C", "age_months", "cycle_count", "soc_dwell_pct", "cell_imbalance_mV", "fade_rate_pct_yr", "soh_24mo_pct"] def write_csv(path="fleet.csv"): with open(path, "w", newline="") as f: w = csv.writer(f) w.writerow(fields) for i in range(N): w.writerow([ f"VIN{i:04d}", int(firmware_buggy[i]), round(float(fast_charge_pm[i]), 2), round(float(ambient_C[i]), 2), round(float(age_months[i]), 1), round(float(cycle_count[i]), 0), round(float(soc_dwell[i]), 1), round(float(cell_imbalance_mV[i]), 1), round(float(degr_rate[i]), 3), round(float(soh_24mo[i]), 2), ]) print(f"wrote {path} ({N} rows)")
Run as a script, this writes fleet.csv and prints the full analysis below it.
The same four moves from the case study, as functions: the correlation screen (§05), the partial-correlation control that exposes the confounder (§07), the regression where the fast-charge coefficient dies, and the firmware stratification that finds the origin (§09).
# === SECTION: analysis (the diagnostic pipeline) ================= features = { "Cell imbalance dV (mV)": cell_imbalance_mV, "Fast-charge events / mo": fast_charge_pm, "Ambient temp (C)": ambient_C, "Cycle count": cycle_count, "High-SOC dwell (%)": soc_dwell, "Age (months)": age_months, } def correlation_screen(y): """Step 1 - rank candidates by raw Pearson correlation.""" print("\n[1] RAW correlation vs fade rate") for name, x in features.items(): r, p = stats.pearsonr(x, y) print(f" {name:24s} r={r:+.3f} p={p:.1e}") def partial_corr(x, y, z): """corr(x, y) with the linear effect of z removed from both. True causes survive this; confounded proxies collapse.""" rx = x - np.polyval(np.polyfit(z, x, 1), z) # residual of x | z ry = y - np.polyval(np.polyfit(z, y, 1), z) # residual of y | z return stats.pearsonr(rx, ry) def partial_screen(y, control, control_name): print(f"\n[2] PARTIAL correlation (controlling for {control_name})") for name, x in features.items(): if x is control: continue r, p = partial_corr(x, y, control) verdict = "survives" if abs(r) > 0.3 and p < 0.05 else "COLLAPSES" print(f" {name:24s} r={r:+.3f} p={p:.2f} {verdict}") def regression(y): """Step 3 - watch the fast-charge coefficient die when dV enters.""" def fit(cols): X = np.column_stack([np.ones(N)] + cols) b, *_ = np.linalg.lstsq(X, y, rcond=None) pred = X @ b r2 = 1 - np.sum((y - pred)**2) / np.sum((y - y.mean())**2) return b, r2 b1, r2_1 = fit([fast_charge_pm, ambient_C]) b2, r2_2 = fit([fast_charge_pm, ambient_C, cell_imbalance_mV]) print("\n[3] REGRESSION") print(f" naive fade~fc+amb : fc_coef={b1[1]:+.3f} R2={r2_1:.3f}") print(f" full fade~fc+amb+dV : fc_coef={b2[1]:+.3f} " f"dV_coef={b2[3]:+.3f} R2={r2_2:.3f}") def stratify(y): """Step 4 - follow the chain upstream to the firmware build.""" bug = firmware_buggy == 1 print("\n[4] FIRMWARE STRATIFICATION") print(f" v3.2.1 (buggy) : dV={cell_imbalance_mV[bug].mean():5.1f} mV " f"fade={degr_rate[bug].mean():.2f} %/yr (n={bug.sum()})") print(f" other builds : dV={cell_imbalance_mV[~bug].mean():5.1f} mV " f"fade={degr_rate[~bug].mean():.2f} %/yr (n={(~bug).sum()})")
The printed output reproduces every number quoted in Parts I and II — fast-charge collapsing to r ≈ −0.09 under control, ΔV holding at ≈ 0.82, the regression coefficient falling from +0.27 to −0.02, and the categorical firmware split. The method recovers the planted truth without ever being told it.