Agentic AI Notes / 01 of 02 / Same vs new data
Did the model learn the underlying pattern, or did it just memorize the answers it was shown?
The idea in one line
Generalization is about performance on data the model has never seen, drawn from the same world it was trained on. It is the gap between how well a model does on its training set and how well it does in the wild.
A model that generalizes has captured the real structure of the problem. A model that fails to generalize has memorized its training examples, including their noise, and falls apart the moment something slightly new arrives.
Think of a student studying for an exam. If they memorize the answers to last year's paper, they ace the practice test and fail the real one. If they learn the concepts, they handle questions they have never seen. Generalization is the difference between those two students.
Why it happens
Every model sits between two failure modes. Underfitting means the model is too simple to capture the pattern, so it does poorly everywhere. Overfitting means the model is flexible enough to trace every wiggle in the training data, including random noise, so it looks brilliant on what it has seen and stumbles on what it has not.
The classic balance is simple to state: a more complex model needs more data to earn its complexity. With too little evidence for too many parameters, the model fills the gaps with noise.
Theory, in more depth
Formally, the object of study is the gap between true risk and empirical risk. For a hypothesis \(h\), a loss \(\ell\), and a sample \(S\) of size \(n\) drawn from distribution \(\mathcal{D}\):
Every theory below is a different strategy for bounding \(R(h)-\hat{R}_S(h)\). They differ in what they blame the gap on: the richness of the model class, the algorithm, or the information the model absorbed.
The classical route bounds the worst-case gap across the whole hypothesis class \(\mathcal{H}\) at once. Rademacher complexity measures how well the class can fit pure random noise:
VC dimension is the distribution-free combinatorial version, giving bounds that scale like \(\tilde{O}(\sqrt{d_{\mathrm{VC}}/n})\). The weakness is built in: a supremum over all of \(\mathcal{H}\) is driven by the most pathological hypothesis, so for heavily overparameterized models these bounds become vacuous.
Margin bounds partly rescue this by rewarding classifiers that separate the data with confidence. The effective complexity becomes the margin-normalized norm of the weights rather than the raw parameter count. For deep networks, spectrally-normalized margin bounds scale with the product of the layer spectral norms divided by the margin, which is why a very large network can still generalize if it separates cleanly with a wide margin.
Instead of one hypothesis, bound the expected risk of a whole distribution \(Q\) over hypotheses, relative to a data-independent prior \(P\) chosen before seeing the data:
This produces the only non-vacuous bounds known for deep networks. The \(\mathrm{KL}\) term measures how far training had to move from the prior, so it connects directly to flat minima: a solution that tolerates weight perturbation needs a smaller \(\mathrm{KL}\), hence a tighter bound. Practical reading: flatter solutions generalize better.
Stability sidesteps the class entirely and asks about the algorithm. If swapping one training point barely changes the output hypothesis (uniform stability \(\beta\)), the gap is \(O(\beta + 1/\sqrt{n})\). This is how stochastic gradient descent earns its generalization: small steps for not-too-many iterations. The information-theoretic view is a cousin: the gap is controlled by the mutual information \(I(S;h)\) between the training set and the learned model, so memorization is penalized literally as information leakage.
The modern puzzle
Classical theory predicts the tidy U-shaped curve above. Reality is stranger. Very large models can fit their training data perfectly, even fit random labels, and still generalize. The U-curve extends into a second descent.
No single explanation covers it. The contributing pieces: the implicit regularization of gradient descent, which drifts toward minimum-norm or maximum-margin solutions; the inductive bias of the architecture itself; and benign overfitting, where high-dimensional structure lets a model absorb noise without being hurt by it. The honest summary is that we have several partial accounts and no unified theory.
The levers
Worked example · Vehicle Health Management
Predict remaining useful life from condition-monitoring features (vibration RMS, bearing temperature, current-signature harmonics). The regressor uses a clipped absolute-error loss \(\ell(f(x),y)=\min(|f(x)-y|,B)\), with horizon cap \(B=1000\) cycles. For a feature map with \(\|\phi(x)\|\le R_\phi\) and a norm-bounded head \(\|w\|\le W\), the per-unit complexity is \(\mathfrak{R}_M(\mathcal{F})\le R_\phi W/\sqrt{M}\), and the bound reads:
200 vehicles run to failure, each logging about 500 sensor snapshots. That is \(N\approx 10^5\) training rows. The tempting move is to plug \(n=N=10^5\) into the bound.
With \(\delta=0.05\), the confidence term scales like \(\sqrt{\log(2/\delta)/(2N)}\approx 0.004\). Times the cap that is a few cycles. It looks fantastic.
The 500 snapshots inside one vehicle's degradation trajectory are heavily correlated, and the label is a deterministic countdown along that trajectory. The exchangeable unit is the vehicle, not the row. The honest sample size is \(M=200\), not \(10^5\).
With \(M=200\), the same term is \(\sqrt{\log(2/\delta)/(2M)}\approx 0.10\), on the order of 100 cycles. The ratio is exactly \(\sqrt{N/M}=\sqrt{500}\approx 22\). That is the entire price of pretending correlated rows are independent.
A live fleet is mostly still running, so RUL labels are right-censored. The evidence that constrains the failure tail is closer to the number of observed failure events. If only 60 of 200 vehicles have failed, the effective \(M\approx 60\) and the term climbs to \(\approx 0.16\). Survival-aware losses (a Cox partial likelihood, or a censoring-weighted pinball loss) keep censored vehicles contributing instead of being dropped.
This is why a model can look excellent offline and disappoint on a brand-new vehicle. The fix at evaluation time is the same lesson the bound teaches: cross-validate by leaving out whole vehicles, never individual rows. And by the PAC-Bayes reading, a flatter, weight-noise-tolerant RUL head carries across model years more gracefully than a sharply fit one.
Measure it in practice
Theory says the gap is governed by independent evidence. The practical translation is one rule: split by the unit that varies, not by the row. For fleet models that unit is the vehicle. Grouped cross-validation produces an estimate whose spread reflects vehicle-to-vehicle variation, which is the variation deployment will actually see.
from sklearn.model_selection import GroupKFold import numpy as np # X: features y: RUL labels vehicle_id: one group label per row gkf = GroupKFold(n_splits=5) errs = [] for tr, te in gkf.split(X, y, groups=vehicle_id): model.fit(X[tr], y[tr]) pred = model.predict(X[te]) errs.append(np.mean(np.abs(pred - y[te]))) # spread across folds reflects vehicle-to-vehicle variation print(np.mean(errs), np.std(errs))
The standard deviation across folds is the honest error bar. If it is large, the model has learned vehicle-specific quirks rather than the shared degradation physics.
The same idea governs an agent's reliability. A tool-use policy that scores well on the traces it was tuned on may have simply memorized those tasks. The honest probe is to hold out whole task families, not individual steps, the agentic analogue of leaving out whole vehicles. A policy that survives unseen task families has learned the skill rather than the script.
References