Catching the Change · Change and fault detection over time

Interactive explorers

Two live tools. Inject a change into a stream and watch the detectors race it, or shift a window's mean and spread and watch which test fires.

Sequential detectors, decide as data arrives

These run online, one sample at a time, and raise an alarm the moment evidence crosses a bound. They are what watches a live signal between scheduled checks.

CUSUM

sequential

Accumulates standardized deviations from a healthy target, so a small persistent step compounds into an alarm. The most sensitive of the classic charts to step changes.

In plain words

A Shewhart limit looks at each sample alone and ignores a shift too small to break it on any single reading. The cumulative sum chart instead adds up how far each sample sits from the healthy mean, after subtracting a small allowance k that absorbs ordinary noise. As long as the process is healthy the sum hovers near zero, reset whenever it would go negative; once a real shift begins, every sample pushes the same way and the running sum climbs steadily until it crosses the decision interval h. Two sums run in parallel, one for upward shifts and one for downward, so either direction trips the alarm.

Health-management example

A converter cooling fan partially seizes and the inlet temperature rises by a fraction of its normal scatter. No single reading looks alarming, but the upward CUSUM accumulates the consistent bias and crosses its limit within a handful of samples, long before a fixed threshold would.

import numpy as np

def cusum(x, target, sd, k=0.5, h=6.0):
    z = (np.asarray(x) - target) / sd
    sh = sl = 0.0
    for i, v in enumerate(z):
        sh = max(0.0, sh + v - k)     # upward
        sl = max(0.0, sl - v - k)     # downward
        if sh > h or sl > h:
            return i                  # alarm index
    return -1

# target and sd come from a healthy reference, never from the live stream
alarm = cusum(stream, target=mu0, sd=sd0, k=0.5, h=6.0)

Practical note

Estimate the target and standard deviation from a verified healthy reference, never from the stream you are watching, or a developing fault inflates the baseline and hides itself. Choose k as half the shift you most want to catch quickly, and set h from the in-control average run length you can tolerate. Reset and re-estimate after a confirmed repair. CUSUM assumes roughly independent, stationary samples; autocorrelation makes it alarm early.

Page (1954), Biometrika 41; Hawkins & Olwell (1998), Cumulative Sum Charts and Charting for Quality Improvement.

EWMA

sequential

An exponentially weighted moving average weights recent samples most, so a gradual drift compounds across the limit. The most sensitive of the charts to slow drifts.

In plain words

The EWMA replaces the raw signal with a running average that forgets the past geometrically: each new value gets weight lambda, the previous estimate keeps the rest. Small lambda means a long memory that smooths hard and reacts to slow drift; large lambda tracks recent values and behaves more like a raw chart. Because the smoothed value has a known steady-state variance, the control limits sit a few of those standard deviations from the target. A slow drift that never trips a per-sample limit gradually pulls the EWMA across its band.

Health-management example

A battery cell's internal resistance creeps up over weeks, nudging its temperature a little higher each cycle. The EWMA of cell temperature bends slowly upward and crosses its limit while a step-tuned CUSUM, waiting for a sharp move, is slower to react.

import numpy as np

def ewma(x, target, sd, lam=0.15, L=3.0):
    width = L * sd * np.sqrt(lam / (2 - lam))
    ucl, lcl = target + width, target - width
    z = target
    for i, v in enumerate(np.asarray(x)):
        z = lam * v + (1 - lam) * z
        if z > ucl or z < lcl:
            return i
    return -1

alarm = ewma(stream, target=mu0, sd=sd0, lam=0.15, L=3.0)

Practical note

Pick lambda for the speed of fault you fear: small (0.05 to 0.2) for slow drifts, larger for abrupt ones; lambda near one reduces to a Shewhart chart. Set limits from a healthy reference, and prefer the time-varying limits for the first few samples. Like CUSUM it assumes independent samples, so whiten or widen the limits for autocorrelated streams.

Roberts (1959), Technometrics 1; Lucas & Saccucci (1990), Technometrics 32.

Page-Hinkley test

sequential

Tracks the cumulative gap between each sample and the running mean, minus a slack, and alarms when it departs from its own running minimum. A popular abrupt-change detector in condition monitoring.

In plain words

The Page-Hinkley test keeps a cumulative sum of how far each sample exceeds the running mean, after subtracting a small tolerance delta that sets how much drift is allowed for free. It also remembers the lowest that cumulative sum has ever been. The alarm is the gap between the current sum and that running minimum: while the process is healthy the sum wanders and the gap stays small, but a genuine upward change makes the sum pull steadily away from its minimum until the gap exceeds a threshold. It is a close relative of CUSUM, framed for streaming data and widely used in condition monitoring.

Health-management example

A bearing begins to spall and the vibration RMS jumps to a new level. The Page-Hinkley statistic, flat through the healthy run, climbs sharply once the jump arrives and crosses its threshold, flagging the abrupt change with a short delay.

import numpy as np

def page_hinkley(x, delta=0.3, threshold=10.0):
    run = cum = ext = 0.0
    for i, v in enumerate(np.asarray(x)):
        run += (v - run) / (i + 1)         # running mean
        cum += (v - run) - delta
        ext = min(ext, cum)
        if cum - ext > threshold:
            return i
    return -1

alarm = page_hinkley(stream, delta=0.3, threshold=10.0)

Practical note

The slack delta sets how much drift is tolerated for free, so tune it to the noise level: too small floods false alarms, too large misses slow faults. Run separate tests for upward and downward changes. Page-Hinkley detects a change but does not size it, so follow an alarm with a window comparison to characterize what changed.

Hinkley (1971), Biometrika 58; Page (1954), Biometrika 41; Gama et al. (2014), ACM Computing Surveys 46 (concept drift).

SPRT (Wald's sequential test)

sequential

Decides between a healthy and a faulty hypothesis with the fewest samples possible, by walking a log-likelihood ratio between two bounds set by the target error rates.

In plain words

Where a fixed-sample test collects a set number of points and then decides, the sequential probability ratio test decides after every sample whether it has enough evidence yet. It accumulates the log of the likelihood ratio between two hypotheses, healthy with mean mu0 and faulty with mean mu1, and compares the running total to two bounds derived from the false-alarm and missed-detection rates you will accept. Cross the upper bound and you declare faulty, cross the lower and you declare healthy, otherwise take another sample. On average it reaches a confident decision with far fewer samples than a fixed test.

Health-management example

An end-of-line rig must pass or fail a unit on a torque measurement without wasting cycles. The SPRT accumulates evidence and, for a clearly faulty unit, crosses the faulty bound in a handful of samples; a borderline unit simply draws a few more before deciding.

import numpy as np

def sprt(x, mu0, mu1, sd, alpha=0.05, beta=0.05):
    A = np.log((1 - beta) / alpha)        # upper bound -> H1
    B = np.log(beta / (1 - alpha))        # lower bound -> H0
    llr = 0.0
    for i, v in enumerate(np.asarray(x)):
        llr += ((mu1 - mu0) / sd**2) * (v - (mu0 + mu1) / 2)
        if llr >= A:
            return "faulty", i
        if llr <= B:
            return "healthy", i
    return "continue", -1

Practical note

The bounds depend on the error rates you choose and on both hypothesized means, so the test is only as good as mu1, the faulty mean you specify; a fault smaller than mu1 takes longer to catch. It assumes independent samples from a known distribution. With a poor variance estimate the decision can be badly miscalibrated, so validate on labeled healthy and faulty runs.

Wald (1945), Ann. Math. Statist. 16; Wald & Wolfowitz (1948), Ann. Math. Statist. 19.

Model-based detection

When you can write down the likelihood of the data under healthy and faulty models, the likelihood ratio is the most powerful test. The challenge is that the fault parameters are usually unknown.

Generalized likelihood ratio test (GLRT)

model-based

The standard model-based test when the fault size and time are unknown. It maximizes the likelihood ratio over every candidate fault parameter, including the change point.

In plain words

A plain likelihood ratio test needs both hypotheses fully specified, but in practice you rarely know the size of a fault or when it started. The generalized likelihood ratio test handles this by plugging in the maximum-likelihood estimate of the unknown parameters under each hypothesis. For a single change in mean at an unknown time, that means scanning every possible change point, fitting the best before-and-after means at each, and keeping the location that maximizes the ratio. The peak value is the test statistic, compared against a threshold from a chi-square approximation or a simulation, and the location of the peak estimates when the change occurred.

Health-management example

A temperature log shows degraded cooling but nobody knows when it began or by how much. The GLRT sweeps every candidate change point, and its statistic peaks sharply at the sample where the regime shifted, both detecting the fault and dating it.

import numpy as np

def glr_mean_shift(x, sd):
    x = np.asarray(x); n = len(x); cs = np.cumsum(x)
    G = np.zeros(n)
    for k in range(1, n):                 # change after index k-1
        n1, n2 = k, n - k
        m1 = cs[k-1] / n1
        m2 = (cs[-1] - cs[k-1]) / n2
        G[k] = (n1 * n2 / n) * (m1 - m2)**2 / sd**2
    return G, int(G.argmax())             # statistic profile, change point

glr, change_point = glr_mean_shift(segment, sd=sd0)

Practical note

Scanning every change point is the cost, and a maximized statistic needs a higher threshold than a fixed chi-square would suggest, so calibrate by simulation or a corrected critical value rather than the naive chi-square quantile. It assumes a known noise model and variance; estimate the variance from healthy data. Near the ends of the window the estimates are noisy, so trim the search range.

Willsky & Jones (1976), IEEE Trans. Automat. Control 21; Basseville & Nikiforov (1993), Detection of Abrupt Changes.

Comparing two windows: classical hypothesis tests

Offline, the question is whether a window of recent data differs from a healthy baseline, and in what way. A mean shift, a variance change, a shape change, and a categorical association are different faults that need different tests.

Mean shifts: t-test and ANOVA

two-window

The t-test asks whether two windows have different means; one-way ANOVA and its F-statistic generalize that to several groups or operating modes at once.

In plain words

The two-sample t-test compares the means of a before and an after window relative to their scatter, answering whether the difference is larger than noise would explain. Use the Welch version, which does not assume the two windows share a variance. When you have more than two groups, say several operating modes or several units, one-way ANOVA tests in a single shot whether any group mean differs, using an F-statistic that compares variation between groups to variation within them. Both assume roughly normal data and are tests of the mean only, blind to a change that leaves the mean fixed.

Health-management example

A pump's discharge pressure is logged before and after a maintenance action. A Welch t-test confirms the mean shifted; across a fleet of pumps in three duty cycles, ANOVA flags that at least one duty cycle now runs at a different mean pressure.

from scipy import stats

# two windows: did the mean move?
t, p = stats.ttest_ind(before, after, equal_var=False)   # Welch

# several groups (operating modes): does any mean differ?
F, p_anova = stats.f_oneway(mode_a, mode_b, mode_c)

Practical note

Use Welch, not the equal-variance t-test, unless you have checked the variances match. Both the t-test and ANOVA assume approximate normality and independent samples, and they test only the mean, so a significant result tells you the mean moved and nothing about spread or shape. With many comparisons, correct for multiple testing.

Welch (1947), Biometrika 34; Fisher (1925), Statistical Methods for Research Workers.

Distribution change: KS and Anderson-Darling

two-window

These compare whole distributions, not just the mean, so they catch a change in shape or spread that a mean test misses entirely.

In plain words

The two-sample Kolmogorov-Smirnov test measures the largest vertical gap between the empirical cumulative distributions of two windows; a large gap means the distributions differ somewhere, in location, spread, or shape. The Anderson-Darling test answers the same question but weights the tails more heavily, so it is more sensitive to changes in the extremes, which often carry the earliest fault signature. Both are distribution-free and need no assumption of normality, which makes them the right reach when you do not know how a fault will reshape the signal.

Health-management example

A signal's mean and variance look unchanged, but a developing fault has made its distribution heavy-tailed, with occasional large excursions. A t-test sees nothing; the KS and especially the tail-weighted Anderson-Darling test flag that the distribution has changed.

from scipy import stats

ks = stats.ks_2samp(before, after)            # largest CDF gap
ad = stats.anderson_ksamp([before, after])    # tail-weighted

print(ks.statistic, ks.pvalue)
print(ad.statistic, ad.pvalue)

Practical note

KS and Anderson-Darling are powerful and assumption-light, but a significant result only says the distributions differ, not how, so follow up with mean and variance tests to characterize the change. They need a decent sample in each window, and like the others assume independent samples; serial correlation inflates significance.

Massey (1951), JASA 46; Scholz & Stephens (1987), JASA 82 (k-sample Anderson-Darling).

Independence: chi-square

two-window

Tests whether two categorical variables are associated, for example whether a fault state depends on the operating mode, by comparing observed counts to what independence would predict.

In plain words

When both variables are categorical, a fault code and an operating mode, a part supplier and a failure outcome, the chi-square test of independence builds a contingency table of counts and compares it to the table you would expect if the two were unrelated. A large chi-square statistic, relative to its degrees of freedom, means the pattern of counts is unlikely under independence, so the variables are associated. It is the categorical counterpart to correlation, and the standard first check for whether a fault clusters in a particular regime or batch.

Health-management example

A fleet logs fault severity against the operating mode each unit spent most time in. The chi-square test shows severity is far from independent of mode, with major faults concentrated in one duty cycle, pointing maintenance at that regime.

from scipy import stats
import numpy as np

# rows = operating mode, cols = fault severity
table = np.array([[42, 9, 4],
                  [15, 31, 8],
                  [6, 12, 35]])
chi2, p, dof, expected = stats.chi2_contingency(table)

Practical note

Each expected cell count should be at least about five, or the chi-square approximation breaks down and you should use Fisher's exact test instead. The test detects association, not its direction or strength, so inspect the residuals to see which cells drive it, and remember a significant association is not causation.

Pearson (1900), Phil. Mag. 50; Agresti (2013), Categorical Data Analysis.

Variance homogeneity: Levene and Bartlett

two-window

These test whether windows share the same variance. A change in spread with no change in mean is a real fault that mean tests are blind to.

In plain words

A loosening mount or a failing controller often shows up first as increased variability, not a shifted average. Levene's test checks for unequal variance by running an analysis of variance on the absolute deviations from each group's center, which makes it robust to non-normal data. Bartlett's test answers the same question and is more powerful when the data really are normal, but is sensitive to departures from it. The plain F-test of the variance ratio is the two-group special case. Run one of these alongside a mean test, because a pure variance change leaves the mean untouched.

Health-management example

An actuator's position error keeps the same average but starts swinging more widely as a bushing wears. The mean is unchanged so a t-test stays silent, while Levene's test on the before and after windows registers the increased variance.

from scipy import stats

lev = stats.levene(before, after)       # robust to non-normality
bart = stats.bartlett(before, after)    # more powerful if normal

# the two-group F-test of the variance ratio
import numpy as np
F = np.var(before, ddof=1) / np.var(after, ddof=1)

Practical note

Prefer Levene over Bartlett unless you are confident the data are normal, since Bartlett is sensitive to non-normality and will flag it as unequal variance. A variance change and a mean change are different faults, so run a variance test alongside a mean test rather than instead of it.

Levene (1960), in Contributions to Probability and Statistics; Bartlett (1937), Proc. R. Soc. A 160.

Decision map

Start from the question you actually have. The badge marks whether the method runs online, is model-based, or compares two windows.

If you are asking...	Reach for	Kind
Is a small persistent step starting in a live signal?	CUSUM	sequential
Is a slow drift building in a live signal?	EWMA	sequential
Did an abrupt change just happen in a stream?	Page-Hinkley	sequential
Decide healthy vs faulty in the fewest samples?	SPRT	sequential
When did it change, and by how much (unknown)?	GLRT	model-based
Did the mean of a window move?	t-test / ANOVA	two-window
Did the whole distribution change?	KS / Anderson-Darling	two-window
Did the variability change but not the mean?	Levene / Bartlett	two-window
Is a fault state associated with an operating mode?	chi-square	two-window

Decision tree

A working order of escalation

Fix the healthy baseline: estimate the in-control mean and standard deviation from a verified healthy reference.
Watch online: CUSUM for steps, EWMA for drifts, Page-Hinkley for abrupt jumps.
For a minimal-sample healthy-versus-faulty call, run an SPRT against a specified faulty model.
When the size and time are unknown, scan with the GLRT to detect and date the change.
Characterize it: compare before and after windows with a mean test, a variance test, and a distribution test together.
At fleet scale, correct for multiple testing and judge alarms by run length, not single crossings.

# common stack
pip install numpy scipy
# cusum, ewma, page_hinkley, sprt, glr_mean_shift, compare_windows all ship in the kit engine

Rules of thumb

Heuristics that hold across most change-detection work, before the method-specific detail.

Match the detector to the fault shape. CUSUM for steps, EWMA for drifts, Page-Hinkley for abrupt jumps; tune each to the smallest change worth catching.
Set limits from healthy data, never from the stream you are watching. A baseline estimated from data that already contains the fault will hide it.
Online finds when, batch says what. A sequential detector flags the change; a two-window test then characterizes it as a mean, variance, or shape change.
Pick the test for the change you fear. A mean test is blind to a pure variance or shape change, so run a variance and a distribution test alongside it.
Judge alarms by run length, not single samples. Report both the in-control average run length and the detection delay when you set a threshold.
Mind independence. CUSUM, EWMA, SPRT and the hypothesis tests all assume independent samples; autocorrelation makes every one of them over-alarm.
Specify the faulty hypothesis honestly. SPRT and the likelihood-ratio tests are only as good as the faulty model you give them; too optimistic a mu1 slows real detection.
Correct for multiple testing at fleet scale. Screening thousands of windows makes some cross any fixed threshold by chance; control the false-discovery rate.

Pick by scenario

Scenario	Pick & why	Kind
Small persistent step in a live signal	CUSUMaccumulates the consistent bias	sequential
Slow drift over many cycles	EWMAlong memory compounds the drift	sequential
Abrupt jump in a monitored stream	Page-Hinkleydeparture from the running minimum	sequential
Pass/fail decision with fewest samples	SPRTstops as soon as evidence suffices	sequential
Change of unknown size and time	GLRTscans every change point	model-based
Did the mean of a window move	Welch t-testtwo-sample mean comparison	two-window
Several modes, any mean differ	ANOVAbetween vs within variation	two-window
Shape or tail changed, mean did not	KS / Anderson-Darlingcompares whole distributions	two-window
Variability changed, mean did not	Levene / Bartletttests equality of variance	two-window
Fault state tied to operating mode	chi-squareassociation in a contingency table	two-window

At a glance

The detectors split first by whether they run online on a stream or compare two fixed windows, then by the kind of change they are tuned for.

Method	Mode	Best for	Output	Key assumption
CUSUM	online	small persistent step	alarm + direction	i.i.d., known healthy mean/sd
EWMA	online	slow drift	alarm	i.i.d., known healthy mean/sd
Page-Hinkley	online	abrupt jump	alarm	roughly stationary baseline
SPRT	online	fewest-sample decision	healthy / faulty	known mu0, mu1, sd
GLRT	batch / online	unknown size & time	statistic + change point	known noise model
t-test / ANOVA	batch	mean shift	statistic + p	approx normal
KS / Anderson-Darling	batch	distribution change	statistic + p	distribution-free
Levene / Bartlett	batch	variance change	statistic + p	Levene robust, Bartlett normal
chi-square	batch	categorical association	statistic + p	expected counts ≥ 5

Which test for which change

What changed	Mean test	Variance test	Distribution test
Mean only	fires	silent	fires
Variance only	silent	fires	fires
Shape / tails only	silent	maybe	fires
Mean and variance	fires	fires	fires

Worked case study: an incipient bearing fault

A bearing begins to wear and its vibration RMS shifts up by less than one standard deviation at sample 250, a change too small for any single-sample limit. Every number below is computed by the engine on the same 420-sample record.

Step 1

Catch it online

With the control limits fixed from a healthy reference, the sequential detectors all fire within a few samples of the change: CUSUM at a delay of 7 samples, EWMA at 5, and Page-Hinkley at 7. None of them would have been triggered by the individual readings, which stay inside their normal range; they each accumulate the small consistent bias instead.

Step 2

Date the change

Knowing that something changed, the GLRT scans every candidate change point and peaks at sample 239, close to the true onset at 250, with no need to have known the shift size in advance. That estimate brackets the window for the characterization step.

Step 3

Say what kind of change

Splitting the record at the change point and comparing the before and after windows tells the rest of the story. The Welch t-test on the mean is decisive (p = 4e-21) and the Kolmogorov-Smirnov test agrees the distribution moved (p = 2e-14), while Levene's test on the variance stays silent (p = 0.95). The signature is a clean mean shift with unchanged spread, exactly what a developing offset fault looks like.

Step 4

Decide with minimum data

Framed as a decision between healthy and faulty, the SPRT on the post-change window reaches accept H1 (faulty) after only 6 samples. That is the value of the sequential framing: a confident call from a handful of readings rather than a fixed-size batch.

Verdict

What the ladder delivered

A sub-sigma shift no single-sample limit could see was caught online within a few samples, dated by the GLRT, characterized as a pure mean shift by the window tests, and confirmed as a fault by the SPRT in a handful of readings. The reflex order is: detect online, date with the GLRT, characterize with the window battery, and decide. Online finds when; batch says what.

Going wider: multivariate change detection

A fleet rarely watches one signal. When sensors are correlated, a small change spreads across several of them and hides under each one's individual limit. The multivariate charts watch the vector as a whole.

MCUSUM and MEWMA

The multivariate CUSUM and multivariate EWMA are the vector versions of the two sequential charts. Each maintains a running vector of accumulated deviations and measures its Mahalanobis length against the healthy covariance, so a coordinated drift across correlated channels registers even when no single channel leaves its band. MCUSUM, like its scalar parent, is tuned for step shifts; MEWMA for slow drifts. They are the time-domain counterpart to the T-squared and Q monitoring of the multivariate companion to this guide.

Practical note

Estimate the mean vector and covariance from a healthy reference, and watch the sample-to-variable ratio: a poorly conditioned covariance makes the Mahalanobis distance unstable, so regularize or reduce dimension first. Set the limit by simulation, since the chi-square approximation is only asymptotic. On an alarm, decompose the statistic back to per-sensor contributions to localize the fault.

Crosier (1988), Technometrics 30 (MCUSUM); Lowry, Woodall, Champ & Rigdon (1992), Technometrics 34 (MEWMA).

Bayesian and multiple-change detection

The GLRT finds one change and gives a hard answer. Two richer formulations relax that: one keeps a full posterior over how long ago the last change was, the other partitions a whole record into many segments at once.

Bayesian online change-point detection

Rather than a single alarm, Bayesian online change-point detection maintains at every step a probability distribution over the run length, the number of samples since the last change. Each new observation either extends the current run, raising its probability, or starts a fresh one, and the posterior is updated in closed form under a conjugate Gaussian model. When a change arrives the mass collapses onto a short run length, which is visible as a sharp reset. The output is a calibrated belief about stability, not just a yes or no, which is valuable when the cost of a false alarm is high.

Practical note

The hazard rate encodes how often you expect changes; set it from domain knowledge, since too low a hazard makes the method sluggish and too high makes it jumpy. The conjugate Gaussian model assumes piecewise-constant Gaussian segments, so heavy tails or trends need a richer model. It is online and recursive, so it suits live monitoring, but its cost grows with the run-length support unless you prune it.

Adams & MacKay (2007), arXiv:0710.3742; Fearnhead & Liu (2007), JRSS-B 69.

Multiple change points with PELT

When a record contains several changes, fitting one at a time is inefficient and biased. PELT finds the whole set at once by minimizing a global cost, the sum of each segment's fitting cost plus a penalty per change point, using exact dynamic programming made linear by a pruning step. With a Gaussian segment cost it responds to a change in mean or in variance, returning the full partition and the segment statistics in one pass.

Practical note

The penalty governs how many change points you get: too small over-segments, too large merges real changes; a BIC-like penalty is a sensible default but validate it against known cases. PELT is offline, for a complete record, not live monitoring. Set a minimum segment length to avoid spurious tiny segments, and match the cost function to the change you expect.

Killick, Fearnhead & Eckley (2012), JASA 107; Truong, Oudre & Vayatis (2020), Signal Processing 167 (review).

Concept drift: the streaming-ML cousins

When the thing being monitored is a data-driven model rather than a sensor, the change of interest is in the model's error stream. The machine-learning community evolved its own detectors for exactly this, close relatives of Page-Hinkley.

DDM, ADWIN, and KSWIN

The Drift Detection Method watches a model's running error rate and raises a warning, then a drift, when the rate climbs significantly above its best-ever level, on the logic that a well-trained model's error should not grow unless the world changed. ADWIN keeps an adaptive window and drops its older part whenever two sub-windows differ by more than a statistical bound, so the window length itself signals drift. KSWIN slides a window and compares recent samples to older ones with a Kolmogorov-Smirnov test, catching distributional drift that a mean-based rate would miss. All three answer the same question for a degrading predictive model that CUSUM and Page-Hinkley answer for a sensor.

Practical note

These run on a model's error or residual stream, so they need ground truth or a proxy to score against. DDM assumes the error rate only improves with learning and can false-alarm on noisy stationary streams, so pair its warning level with a confirmation. Tune the window and confidence to trade detection delay against false alarms, exactly as with the classical charts. On a drift, the usual response is to retrain or adapt the model.

Gama et al. (2004), SBIA (DDM); Bifet & Gavalda (2007), SDM (ADWIN); Raab, Heusinger & Schleif (2020), Neurocomputing 416 (KSWIN).

Designing for run length

Every sequential detector trades two quantities: how long it runs before a false alarm, and how fast it reacts to a real change. Tuning is choosing where on that trade-off to sit, and it cannot be skipped.

Average run length, and the tuning it implies

The right figure of merit for a sequential chart is not a per-sample false-positive rate but the average run length: the mean number of samples until an alarm. In control, that is the time between false alarms and you want it long; after a change, it is the detection delay and you want it short. The allowance and limit set both at once. A tighter limit shortens the delay but also shortens the in-control run length, so it false-alarms more often; a larger allowance targets a bigger shift. There is no setting that improves both, so design to a target in-control run length and accept the delay that follows.

Practical note

Fix the in-control run length first, from how many false alarms per month the operation can absorb, then read off the detection delay for the smallest fault you care about. Estimate both by simulation on healthy data rather than trusting a formula, especially when samples are autocorrelated, which always shortens the true in-control run length below the nominal.

Page (1954), Biometrika 41; Montgomery (2013), Introduction to Statistical Quality Control.

Evaluating a detector honestly

A single number, one threshold's delay or one false-alarm count, tells you almost nothing. A detector is a curve, and detectors should be compared as curves on labeled data.

The delay versus false-alarm operating curve

Sweep a detector's threshold and, at each setting, measure two things on labeled runs: the false-alarm rate, the fraction of healthy runs that alarm before any change, and the detection delay, the mean lag after a real change. Plotting one against the other traces an operating curve, the change-detection analogue of an ROC. A detector whose curve sits below and to the left of another is better at every operating point; where curves cross, the choice depends on which error is costlier. This is the only fair way to compare CUSUM against Page-Hinkley against a learned detector, and it requires labeled run-to-failure data to be meaningful.

Practical note

Evaluate on held-out runs, never the data used to set limits, and report the delay at a fixed, operationally acceptable false-alarm rate rather than a best case. Detection delay is more honest than raw accuracy for streaming faults, since a detector that eventually alarms is not the same as one that alarms in time. Benchmarks such as the Numenta Anomaly Benchmark formalize this with delay-weighted scoring.

Basseville & Nikiforov (1993), Detection of Abrupt Changes; Lavin & Ahmad (2015), IEEE ICMLA (NAB).

Beyond a mean step

Most of this guide tracks a shift in the mean. Real faults also change the variance, arrive in autocorrelated streams that break the independence assumption, or hide in the frequency content. Three adjustments cover those.

A CUSUM for variance

A loosening mount or a failing controller often increases variability while leaving the mean fixed, and a mean CUSUM is blind to it. Running a CUSUM on the standardized squared deviation, which has a known in-control expectation, accumulates a genuine increase in spread into an alarm. Run it alongside the mean CUSUM so a change in either is caught.

Practical note

The squared deviation is right-skewed, so a variance CUSUM needs a higher limit than the mean version for the same in-control run length; calibrate it on healthy data. The same idea extends to monitoring an event rate with a Poisson CUSUM.

Hawkins & Olwell (1998), Cumulative Sum Charts and Charting for Quality Improvement.

Autocorrelation breaks the limit

CUSUM, EWMA and the hypothesis tests all assume independent samples. Real sensor streams are autocorrelated, which makes the statistic wander and a naive chart cross its limit with no change present at all. The fix is to model the correlation, fit a simple autoregression and monitor the residual, which is close to independent, so the limits become valid again. The alternative is to widen the limits or set them empirically to respect the correlation.

Practical note

Check the residual's autocorrelation after fitting; if structure remains, add autoregressive order. The same caution applies to every test in this guide, so when a stream is sampled fast enough to be autocorrelated, whiten it or judge alarms by run length rather than single crossings.

Montgomery & Mastrangelo (1991), J. Quality Technology 23.

Spectral change

A bearing defect or a resonance announces itself as a new tone, a change in the frequency content, while the overall amplitude and variance barely move. Detection in the time domain misses it. The remedy is to compute a feature, the power in a diagnostic frequency band on a sliding window, and run a change detector on that feature stream instead of the raw signal. The mean step the detectors are good at reappears, now in the right coordinate.

Practical note

Choose the band from the physics, for example a bearing's characteristic defect frequency, and the window long enough to resolve it but short enough to react. Order-track against shaft speed for rotating machinery so a speed change is not mistaken for a fault. Any of the sequential detectors then applies to the feature.

Randall & Antoni (2011), Mech. Syst. Signal Process. 25 (bearing diagnostics).

References

Page (1954), Biometrika 41; Hawkins & Olwell (1998), Cumulative Sum Charts and Charting for Quality Improvement.
Roberts (1959), Technometrics 1; Lucas & Saccucci (1990), Technometrics 32.
Hinkley (1971), Biometrika 58; Page (1954), Biometrika 41; Gama et al. (2014), ACM Computing Surveys 46 (concept drift).
Wald (1945), Ann. Math. Statist. 16; Wald & Wolfowitz (1948), Ann. Math. Statist. 19.
Willsky & Jones (1976), IEEE Trans. Automat. Control 21; Basseville & Nikiforov (1993), Detection of Abrupt Changes.
Welch (1947), Biometrika 34; Fisher (1925), Statistical Methods for Research Workers.
Massey (1951), JASA 46; Scholz & Stephens (1987), JASA 82 (k-sample Anderson-Darling).
Pearson (1900), Phil. Mag. 50; Agresti (2013), Categorical Data Analysis.
Levene (1960), in Contributions to Probability and Statistics; Bartlett (1937), Proc. R. Soc. A 160.

Take it into your editor

The same content is packaged as a VS Code Copilot skill in the progressive-disclosure layout: a routing SKILL.md, per-topic reference docs, a tested change_detection.py engine that implements every method with numpy and scipy only, and Copilot integration files (a change-detection analyst chat mode, a /detect prompt, and path-scoped instructions). Drop the folder into your kit, or point Copilot at the skill directly.

↓ change-detection-kit.zip
SKILL.md · references · change_detection.py · Copilot agent + prompt

Link resolves when the zip sits beside this file. Run python scripts/change_detection.py for the self-test on synthetic change data.

Changing now?

When and how big?

What kind?

Race the online detectors

Two windows, which test fires?

CUSUM

In plain words

EWMA

In plain words

Page-Hinkley test

In plain words

SPRT (Wald's sequential test)

In plain words

Generalized likelihood ratio test (GLRT)

In plain words

Mean shifts: t-test and ANOVA

In plain words

Distribution change: KS and Anderson-Darling

In plain words

Independence: chi-square

In plain words

Variance homogeneity: Levene and Bartlett

In plain words

Decision map

Decision tree

A working order of escalation

Rules of thumb

At a glance

Catch it online

Date the change

Say what kind of change

Decide with minimum data

What the ladder delivered

MCUSUM and MEWMA

Joint chart vs. single-sensor limits

Bayesian online change-point detection

Multiple change points with PELT

DDM, ADWIN, and KSWIN

Average run length, and the tuning it implies

ARL design calculator

The delay versus false-alarm operating curve

A CUSUM for variance

Autocorrelation breaks the limit

Spectral change

References

Take it into your editor