Reading the Residual

A Luenberger observer estimates the state from the model and the measurements; the leftover output error is the residual. By placing the error-dynamics poles, the designer fixes how fast it settles, and the fault detection filter shapes which faults appear in which directions.

In plain words

The parity space builds a residual from a window of data; an observer builds one recursively. It runs a model of the plant in parallel with the real system, correcting its state estimate by a gain times the output error, and that output error, the innovation, is the residual. Pole placement sets the eigenvalues of the estimation-error dynamics, trading settling speed against noise sensitivity. With the model correct the residual is near zero plus noise; a fault the observer does not model drives it away from zero. The Beard-Jones fault detection filter is the same idea with the gain chosen so that each fault forces the residual into a fixed, known direction, which turns detection into isolation.

# see scripts/residuals.py
L = luenberger_observer(A, C, poles=[0.2, 0.25])
res = observer_residual(A, B, C, L, u, y)["residual"]
# residual ~ noise under the model, steps away at a fault

Luenberger (1971), IEEE TAC 16; Beard (1971) and Jones (1973); Chen & Patton (1999), Robust Model-Based FDI.

An observer designed so its residual is decoupled from a disturbance that enters through a known direction. The residual then responds to faults but not to the disturbance, the classic route to a robust residual that does not false-alarm on a modelled nuisance input.

In plain words

A plain observer reacts to everything the model omits, including disturbances that are not faults, so it false-alarms whenever a known disturbance acts. The unknown input observer removes that sensitivity by construction. Given the direction through which the disturbance enters the state, it transforms the estimation so the disturbance is annihilated, leaving error dynamics that the designer can still stabilize by pole placement. The residual is then provably insensitive to that disturbance while remaining sensitive to faults in other directions. The price is an existence condition, the disturbance must be observable in a particular sense, rank of the output times the disturbance direction equal to the disturbance dimension, and detectability of the decoupled pair.

# see scripts/residuals.py (needs rank(C E) = rank(E))
uio = unknown_input_observer(A, B, C, E, poles=[0.2, 0.25])
res = uio_residual(uio, u, y)["residual"]
# flat under the disturbance E, responsive to faults

Chen & Patton (1999), Robust Model-Based FDI; Darouach et al. (1994), IEEE TAC 39.

A bank of observers, each driven by a different subset of the sensors, so a fault in one sensor disturbs every observer except the one that ignores it. The pattern of which residuals fire isolates the faulty sensor, turning a bank of detectors into an isolator.

In plain words

One residual detects; isolation needs several with a known fault-to-residual pattern. The generalized observer scheme builds that pattern structurally. It runs one observer per sensor, each using all the outputs except its own, so observer i is by construction insensitive to a fault in sensor i but sensitive to faults in any other sensor. When sensor i fails, every observer except the i-th sees the corrupted signal and its residual fires, while observer i stays clean, so the one quiet residual names the fault. The dedicated observer scheme is the dual arrangement, each observer driven by a single sensor. Both convert a set of observers into a structured-residual isolator without a hand-built signature table.

# see scripts/residuals.py
gos = generalized_observer_scheme(A, B, C, poles=[0.2, 0.25], u=u, y=y)
gos["norms"]      # residual norm per observer
gos["cleanest"]   # the observer that excludes the faulty sensor

Clark (1978), IEEE TAES; Frank (1990), Automatica 26; Chen & Patton (1999).

A matched detector for an abrupt change of unknown size and onset in the residual mean. It maximizes the log-likelihood ratio over all recent onset times, giving a statistic that is quiet under the no-fault model and spikes once a bias appears, the residual-domain bridge to your change-detection methods.

In plain words

A fixed threshold on the raw residual ignores that a fault is often a structured change, a step in the mean, not just a single large sample. The generalized likelihood ratio test exploits that structure. For every possible onset time in a recent window it computes how much more likely the data are under a hypothesized step than under no change, and takes the maximum, so it is matched to a step of any size starting at any time. The statistic stays near zero while the residual is white and rises sharply once a sustained bias accumulates, with a chi-square threshold from the likelihood theory. It is the same sequential-detection logic as the CUSUM in your change-detection work, specialized to a residual with known noise covariance.

# see scripts/residuals.py
glr = glr_detector(residual, sigma=sigma_healthy, m_max=40)
fault = glr["glr"] > glr["threshold"]   # spikes after an abrupt bias

Willsky & Jones (1976), IEEE TAC 21; Basseville & Nikiforov (1993), Detection of Abrupt Changes.

A threshold that scales with the operating point. Model uncertainty makes the healthy residual grow with load, so a fixed threshold misses faults at low load or false-alarms at high load; an adaptive threshold tracks the load and holds the false-alarm rate roughly constant.

In plain words

In practice the no-fault residual is never exactly zero, because the model is never exact, and its spread typically grows with the input or load, the larger the excitation the larger the modelling error. A single fixed threshold cannot serve the whole envelope: set it for high load and small faults at low load slip through, set it for low load and normal high-load operation trips it. An adaptive threshold makes the bound a function of the operating point, widening with load by an amount tuned to the model uncertainty, so the effective false-alarm rate stays roughly constant across conditions. It is the simplest form of the robustness that the unknown input observer pursues structurally.

# see scripts/residuals.py
at = adaptive_threshold(residual, u=load, base=0.2, k_u=0.15)
alarm = at["alarm"]      # |residual| > base + k_u*|load|

Emami-Naeini et al. (1988), Automatica 24; Ding (2013), Model-Based Fault Diagnosis Techniques.

A deterministic alternative to statistical thresholds. Given bounded process and measurement disturbances, it propagates a guaranteed envelope for the residual; while the measurement stays inside, no fault is possible, so an exceedance is a guaranteed detection rather than a probable one.

In plain words

Statistical thresholds accept a false-alarm rate and a missed-detection rate; sometimes you want a guarantee instead. Set-membership detection provides one by replacing probability distributions with hard bounds. If the process and measurement disturbances are known to lie within stated limits, the no-fault residual must lie within an envelope that those limits imply, computed by propagating the bounds through the observer error dynamics to a steady-state interval. As long as the measurement stays inside the envelope, the data are consistent with the healthy model and no fault has occurred; the moment it steps outside, a fault is certain, not merely likely. The cost is conservatism, the bound is wide, so only sufficiently large faults are guaranteed to be caught.

# see scripts/residuals.py
iv = interval_observer(A, B, C, L, u, y, w_bound=0.06, v_bound=0.15)
alarm = iv["alarm"]      # leaving iv["envelope"] is a guaranteed detection

Milanese et al. (1996), Bounding Approaches to System Identification; Puig (2010), Int. J. Adapt. Control Signal Process.

Estimate a fault's magnitude by adding it to the state vector as an extra, slowly-varying state and running a Kalman filter on the enlarged model. The estimated fault state tracks an additive bias, turning detection into identification with no new machinery.

In plain words

The most direct way to estimate a fault is to make it part of what the filter estimates. Augment the state vector with the fault, model it as nearly constant (a slow random walk), and write down how it enters the dynamics or the measurements. A Kalman filter on this augmented model then estimates the fault alongside the true state, so its dedicated component reads off the fault magnitude in real time. It reuses the entire Kalman machinery, including the innovation as a residual, and degrades gracefully, the fault estimate is smoothed by the filter rather than jumping on every sample. The assumption is that the fault is slow relative to the dynamics, which holds for biases, drifts, and developing faults but not for instantaneous jumps.

# see scripts/residuals.py
af = augmented_fault_kalman(A, B, C, Q, R, fault_dir=B.ravel(), y=y, u=u)
af["fault"]      # estimated fault magnitude over time

Friedland (1969), IEEE TAC 14; Gillijns & De Moor (2007), Automatica 43.

A proportional-integral observer adds an integral of the output error that accumulates into a fault estimate, reconstructing an additive fault online. The sliding-mode observer reconstructs it instead from the equivalent output-error injection, both recovering the fault signal rather than merely flagging it.

In plain words

A standard observer uses the output error proportionally to correct the state. A proportional-integral observer adds a second loop that integrates that error into an explicit fault estimate, so a persistent residual is driven into a reconstructed fault signal instead of biasing the state, much as integral action in a controller cancels a steady offset. The result tracks a developing fault, a drift or a ramp, in real time. The sliding-mode observer reaches the same goal by a different mechanism: it forces the estimation error to zero with a switching injection, and the average of that injection, the equivalent output-error injection, reconstructs the fault. Both turn the residual into an estimate of the fault itself, the basis for fault-tolerant control.

# see scripts/residuals.py
pio = pi_observer(A, B, C, L, Kf=[0, 0.18], fault_dir=B.ravel(), u=u, y=y)
pio["fault"]     # reconstructed fault signal

Beale & Shafai (1989), Int. J. Control; Edwards, Spurgeon & Patton (2000), Automatica 36.

A bank of Kalman filters, one per hypothesized mode, healthy and one or more faulty, mixed each step by a Markov transition matrix. The posterior probability of each mode both detects and isolates: a rising probability for a faulty model names the fault, tying this to your Bayesian toolkit.

In plain words

When a system can be in one of several regimes, healthy or various faulty ones, a single model cannot serve all of them. The interacting multiple model estimator runs a Kalman filter for each candidate mode and maintains a probability over which mode is active, governed by a Markov chain of mode transitions. At each step it mixes the filters' estimates according to those probabilities, updates each filter, and reweights the mode probabilities by how well each predicted the new measurement. The output is a soft, continuously updated belief over modes, so the probability of a faulty mode rising toward one is simultaneously a detection and an isolation, and the estimate stays good across the switch because the filters are blended rather than chosen abruptly.

# see scripts/residuals.py
m0 = dict(F=A, H=C, Q=Q, R=R_healthy, x0=x0, P0=P0)
m1 = dict(F=A, H=C, Q=Q, R=R_faulty,  x0=x0, P0=P0)
imm = imm_filter(y, [m0, m1], Pi=[[0.97, 0.03], [0.03, 0.97]], u=u, B=B)
imm["mode_prob"]    # P(faulty mode) rises at the fault

Blom & Bar-Shalom (1988), IEEE TAC 33; Mazor et al. (1998), IEEE TAES 34.

Bartlett's whiteness test in the frequency domain. The normalized cumulative sum of the residual's periodogram follows the diagonal for white noise; periodic or colored structure makes it bow away beyond a Kolmogorov band, catching narrowband content the autocorrelation can smear.

In plain words

The autocorrelation tests look at the residual in the time domain and can blur a concentrated periodic component across several lags. The cumulative periodogram looks in the frequency domain instead. White noise has a flat spectrum, so the running cumulative sum of its periodogram rises in a straight diagonal line from zero to one across frequency. A periodic or colored residual has its power concentrated at some frequencies, so the cumulative curve climbs steeply there and bows away from the diagonal, and a Kolmogorov-type band says how far it may stray under whiteness before the departure is significant. It is especially good at exposing a single tonal component, a once-per-revolution leakage, that a time-domain test would dilute.

# see scripts/residuals.py
cp = cumulative_periodogram(residual)
cp["white"]      # False if the curve bows outside the band

Bartlett (1955), Stochastic Processes; Priestley (1981), Spectral Analysis and Time Series.

Engle's test for a time-varying residual variance. A residual can pass every mean-whiteness test yet have its variance cluster or grow, the signature of an intermittent or developing fault; regressing the squared residual on its own lags exposes it.

In plain words

The whiteness tests check that the residual has no correlation in its mean, but they say nothing about its variance, which can change over time even when the mean stays white. Such conditional heteroscedasticity, volatility that clusters, is itself a fault signature: an intermittent fault, a loosening part, or a developing wear process inflates the residual's spread in bursts. Engle's test detects it by regressing the squared residual on its own recent past; if the squared residual is predictable from its history, its variance is time-varying, and the test's statistic, the sample size times the regression's goodness of fit, is large with a small p-value. It catches a class of faults the mean-based tests are blind to.

# see scripts/residuals.py
at = arch_test(residual, lags=5)
at["heteroscedastic"], at["pvalue"]   # variance change in the residual

Engle (1982), Econometrica 50; Bollerslev (1986), J. Econometrics 31.

The Brown-Durbin-Evans cumulative-sum test for a slow drift in the residual mean. The running sum of standardized residuals stays within a widening boundary if the model holds throughout; crossing it signals that the mean drifted, a slow fault or a model that has stopped fitting.

In plain words

A fault need not be abrupt; a residual mean can creep away from zero so gradually that no single sample looks wrong and a whiteness test on a short window sees nothing. Accumulating the residual exposes the creep. The Brown-Durbin-Evans test forms the cumulative sum of standardized residuals and compares its path to a pair of boundary lines that widen with time to keep a constant false-alarm rate under stability. While the residual is genuinely zero-mean the cumulative sum wanders like a bounded random walk and stays inside; once a persistent bias sets in, the sum drifts steadily and eventually breaches a boundary, dating the loss of fit. It is the model-validation counterpart of the CUSUM change detectors, watching for structural drift rather than an abrupt jump.

# see scripts/residuals.py
cu = cusum_residual(residual)
cu["stable"]     # False if the path breaches the boundary (drift)

Brown, Durbin & Evans (1975), J. R. Stat. Soc. B 37; Ploberger & Kramer (1992), Econometrica 60.

Parity relations identified directly from healthy input-output data rather than from a state-space model. The left null space of a stacked data matrix gives generators that annihilate normal data, so applied to new data they yield a residual that grows under a fault, the model-free counterpart to the parity space.

In plain words

The classical parity space needs a state-space model to compute the relations that should hold among inputs and outputs. When such a model is unavailable or untrustworthy, the same relations can be learned from data. Stacking windows of inputs and outputs from healthy operation into a matrix, its dominant directions describe how normal data move, and its smallest directions, the left null space, are combinations that are essentially constant, zero up to noise, for any healthy window. Those combinations are the data-driven parity relations: apply them to a fresh window and the result stays near zero in health and departs under a fault. It is identification and residual design in one, closely related to subspace system identification, and it inherits the parity space's clean fault sensitivity without ever writing down a model.

# see scripts/residuals.py
train = dd_parity(U_healthy, Y_healthy, s=2)
test = dd_parity(U_new, Y_new, s=2, W=train["W"], mean=train["mean"])
test["residual"]      # near zero in health, jumps at a fault

Ding (2014), Data-Driven Design of Fault Diagnosis Systems; Qin (2012), Annu. Rev. Control 36.

A residual from a learned model of normal data: project onto the healthy subspace and reconstruct, and the reconstruction error is the residual. Small for normal operation, large when a fault pushes data off the learned manifold, it is the squared-prediction-error of the multivariate monitoring world.

In plain words

If the healthy data live near a low-dimensional surface, a model of that surface gives a residual for free. Principal component analysis finds the leading linear subspace of the healthy data; projecting a new sample onto it and reconstructing leaves an error, the part of the sample that does not lie on the healthy manifold. That reconstruction error, the squared-prediction-error or Q statistic, is small for normal operation and large when a fault moves the data off the manifold, so it is a residual built entirely from data. An autoencoder generalizes the idea to a nonlinear manifold, learning an encoder and decoder whose reconstruction error plays the same role, but the linear version already connects this directly to the Q statistic in your multivariate-monitoring toolkit.