Reading the Couplings

Linear coupling between two signals; the baseline for sensor-pair relationships.

In plain words

Pearson r asks one thing: when one signal goes up by a fixed amount, does the other go up (or down) by a roughly fixed amount too? It is the slope of the best straight line, scaled to sit between minus one and plus one. It only sees straight-line relationships, so a strong curved link can still give r near zero.

import numpy as np
from scipy.stats import pearsonr

r, p = pearsonr(sensor_A, sensor_B)
print(f"Pearson r = {r:.3f}  (p = {p:.1e})")

Pearson, K. (1896). Mathematical contributions to the theory of evolution III. Phil. Trans. R. Soc. A 187.

Rank-based, robust to nonlinearity and outliers; good when the monotonic trend matters more than the exact shape.

In plain words

Instead of the raw values, these use the order (1st, 2nd, 3rd ...). Spearman is Pearson computed on ranks; Kendall's tau counts how often two points agree on direction (both up, or both down). Because ranks ignore the exact spacing, a curved-but-always-rising relationship scores near one, and a single wild outlier barely moves the result.

from scipy.stats import spearmanr, kendalltau

rho, _ = spearmanr(load, wear)     # Pearson on the ranks
tau, _ = kendalltau(load, wear)    # concordant vs discordant pairs
print(f"Spearman {rho:.3f}   Kendall {tau:.3f}")

Spearman (1904), Am. J. Psychol. 15; Kendall (1938), Biometrika 30.

Association between two variables after regressing out a controlling set; the key tool for separating direct coupling from shared-driver effects.

In plain words

Two signals can look coupled only because a third thing drives them both. Partial correlation strips the influence of that third set out of BOTH signals first, then measures what correlation is left. If the link survives, it is more likely a direct one; if it collapses, the third variable was the real story.

import pandas as pd, pingouin as pg

df = pd.DataFrame({"A": sensor_A, "B": sensor_B,
                   "load": load, "temp": ambient})
# remove load and temp from BOTH A and B, then correlate
res = pg.partial_corr(data=df, x="A", y="B", covar=["load", "temp"])
print(res[["r", "p-val"]])

Baba, Shibata & Sibuya (2004). Partial correlation and conditional correlation as measures of conditional independence. Aust. N. Z. J. Stat. 46.

Removes the confounders from only one variable; useful for attributing incremental explanatory power.

In plain words

Same idea as partial correlation, but the controlling set is removed from just one of the two variables, not both. That answers a subtly different question: how much UNIQUE information does this one predictor add about the target, beyond what the controls already explain? It is the natural language of incremental contribution.

import pingouin as pg
# x_covar removes controls from X (the predictor) ONLY
res = pg.partial_corr(data=df, x="vibration", y="fault_idx",
                      x_covar=["load", "temp"])
print("unique contribution:", res["r"].iloc[0])

Cohen, Cohen, West & Aiken (2003). Applied Multiple Regression/Correlation Analysis, 3rd ed.

Adds time-shift, so you recover propagation delay and direction of influence between subsystems.

In plain words

Ordinary correlation lines up two signals at the same instant. Cross-correlation slides one signal in time and measures correlation at every shift. The shift where the match is strongest is the delay between them, and its sign tells you which signal moved first.

import numpy as np
from scipy import signal

a = a - a.mean(); b = b - b.mean()
xc   = signal.correlate(b, a, mode="full")
lags = signal.correlation_lags(len(b), len(a), mode="full")
delay = lags[np.argmax(xc)]
print(f"B lags A by {delay} samples  (>0 means A leads)")

Box & Jenkins (1976), Time Series Analysis; Knapp & Carter (1976), IEEE Trans. ASSP 24.

Capture nonlinear and non-monotonic dependence that Pearson misses.

In plain words

Some relationships are real but not straight or even always-rising, think of a U-shape, or points scattered on a ring. Distance correlation is zero only when two variables are truly independent, so it catches these. MIC (maximal information coefficient) scans many grid resolutions to score how cleanly one variable carves up the other, again catching curves and clusters.

import dcor
from minepy import MINE

dc = dcor.distance_correlation(x, y)   # 0 iff independent
m = MINE(); m.compute_score(x, y)
print(f"dCor {dc:.3f}   MIC {m.mic():.3f}")

Szekely, Rizzo & Bakirov (2007). Measuring and testing dependence by correlation of distances. Ann. Statist. 35; Reshef et al. (2011), Science 334 (MIC).

Information-theoretic dependence; conditional MI is the nonlinear analogue of partial correlation for fault isolation.

In plain words

Mutual information measures how many bits knowing one signal saves you when guessing the other, of any shape of relationship, not just linear. Conditional mutual information asks the same thing AFTER you already know a third signal: if it drops to near zero, the third signal explained the link. That is the nonlinear cousin of partial correlation.

from sklearn.feature_selection import mutual_info_regression
from npeet import entropy_estimators as ee   # kNN estimators

mi  = mutual_info_regression(x.reshape(-1, 1), y)[0]
cmi = ee.cmi(x.reshape(-1, 1), y.reshape(-1, 1), z.reshape(-1, 1))
print(f"MI {mi:.3f}   conditional MI I(X;Y|Z) {cmi:.3f}")

Kraskov, Stogbauer & Grassberger (2004), Phys. Rev. E 69; Frenzel & Pompe (2007), Phys. Rev. Lett. 99 (conditional MI).

Frequency-domain correlation; tells you at which frequencies two signals are coupled.

In plain words

Coherence is correlation computed frequency by frequency. It runs from zero to one and answers: at this particular vibration frequency, do the two signals rise and fall in lockstep? Two signals can be uncorrelated overall yet strongly coherent in one narrow band.

from scipy.signal import coherence

f, Cxy = coherence(sig_a, sig_b, fs=fs, nperseg=1024)
peak = f[Cxy.argmax()]
print(f"strongest coupling at {peak:.1f} Hz  (coherence {Cxy.max():.2f})")

Carter (1987), Proc. IEEE 75; Bendat & Piersol, Random Data.

Agreement among repeated or redundant measurements of the same quantity; the right metric for triple-redundant sensor consistency, not Pearson (which ignores systematic offset).

In plain words

When several sensors are supposed to measure the SAME thing, you do not just want their shapes to match, you want their numbers to match. Pearson happily reports r equal one even if one sensor reads a constant ten degrees high. ICC treats that constant offset as disagreement, which is what redundancy management actually needs.

import pingouin as pg
# long format: one row per (sample, sensor, reading)
icc = pg.intraclass_corr(data=long_df, targets="sample",
                         raters="sensor", ratings="value")
print(icc[icc.Type == "ICC2"][["ICC", "CI95%"]])

Shrout & Fleiss (1979), Psychol. Bull. 86; McGraw & Wong (1996), Psychol. Methods 1.

Agreement vs. a reference, penalizing both scatter and bias; ideal for sensor-vs-truth calibration-drift monitoring.

In plain words

CCC measures how close paired readings sit to the perfect y equals x line, not just how tightly they cluster around SOME line. It combines two penalties: random scatter, and any systematic gap (bias). A sensor that is precise but biased gets a high Pearson and a low CCC, which is the honest verdict.

import numpy as np
def ccc(x, y):
    cov = np.cov(x, y, bias=True)[0, 1]
    return 2 * cov / (x.var() + y.var() + (x.mean() - y.mean()) ** 2)
print("CCC =", ccc(reference, sensor))

Lin, L. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics 45.

Agreement among multiple rankers/sensors when only ordering is trusted.

In plain words

Kendall's W rates how much several rankers agree on an ordering, from zero (no agreement) to one (identical rankings). Use it when the absolute numbers are not comparable but the ORDER is, for example several diagnostics each ranking suspects.

import numpy as np
def kendall_w(ranks):                 # ranks: raters x items
    m, n = ranks.shape
    R = ranks.sum(axis=0)
    S = ((R - R.mean()) ** 2).sum()
    return 12 * S / (m ** 2 * (n ** 3 - n))
print("Kendall W =", kendall_w(rank_matrix))

Kendall & Babington Smith (1939). The problem of m rankings. Ann. Math. Statist. 10.

Association between categorical variables; fault-code co-occurrence, fault-mode vs. operating-mode tables.

In plain words

The chi-square test asks whether two labels occur together more (or less) often than chance. Cramer's V turns that test into a strength score between zero and one, so you can compare associations across different tables, the categorical answer to 'how strongly related are these?'.

import numpy as np
from scipy.stats import chi2_contingency

chi2, p, dof, _ = chi2_contingency(table)     # DTC x mode counts
n = table.sum(); r, k = table.shape
V = np.sqrt(chi2 / (n * (min(r, k) - 1)))
print(f"Cramer's V = {V:.3f}  (p = {p:.1e})")

Cramer (1946), Mathematical Methods of Statistics; Bergsma (2013), J. Korean Statist. Soc. 42 (bias correction).

Binary-binary and continuous-binary association (a continuous signal vs. fault-present flag).

In plain words

Phi is correlation between two yes/no variables (a tidy 2x2 case). Point-biserial is correlation between a continuous measurement and a yes/no label, mathematically just Pearson with one side coded as 0 and 1. It tells you how cleanly a numeric feature separates the two classes.

from scipy.stats import pointbiserialr

r_pb, p = pointbiserialr(fault_flag, vibration_rms)   # flag in {0,1}
print(f"point-biserial r = {r_pb:.3f}")
# phi for a 2x2 table:  phi = sqrt(chi2 / n)

Tate, R. F. (1954). Correlation between a discrete and a continuous variable. Ann. Math. Statist. 25.

Directional categorical dependence; how much knowing the operating mode reduces uncertainty about the fault state.

In plain words

Theil's U measures, in information terms, how much knowing one label shrinks your uncertainty about another, and crucially it is directional. U(fault | mode) can differ from U(mode | fault). That asymmetry is exactly what you want when one variable is a cause-like driver and the other a consequence.

import numpy as np
from collections import Counter
def entropy(v):
    p = np.array(list(Counter(v).values()), float); p /= p.sum()
    return -(p * np.log(p)).sum()
def theils_u(x, y):                       # U(x | y): does y predict x?
    Hx = entropy(x)
    Hxy = sum((c / len(y)) * entropy([x[i] for i in range(len(y)) if y[i] == yv])
              for yv, c in Counter(y).items())
    return (Hx - Hxy) / Hx
print("U(fault|mode) =", theils_u(fault, mode))

Theil, H. (1972). Statistical Decomposition Analysis; Press et al., Numerical Recipes.

Latent-continuous correlation behind thresholded or ordinal indicators.

In plain words

Sometimes a 0/1 flag (or an ordinal level) is really a hidden continuous quantity that crossed a threshold. Tetrachoric (for binary) and polychoric (for ordinal) estimate the correlation of those HIDDEN continuous variables, undoing the information you lost by thresholding. The result is usually larger and more honest than treating the flags as plain numbers.

import numpy as np
from scipy.stats import norm, multivariate_normal as mvn
from scipy.optimize import brentq
def tetrachoric(t):                       # t = [[n11,n10],[n01,n00]]
    n = t.sum()
    hx = norm.ppf(t[0].sum() / n); hy = norm.ppf(t[:, 0].sum() / n)
    f = lambda r: mvn([0, 0], [[1, r], [r, 1]]).cdf([hx, hy]) - t[0, 0] / n
    return brentq(f, -0.999, 0.999)
print("tetrachoric r =", tetrachoric(np.array([[40, 12], [10, 38]])))

Pearson (1900), Phil. Trans. R. Soc. A 195; Olsson (1979), Psychometrika 44 (polychoric).

Kernel independence test; a robust general-purpose nonlinear dependence detector and a standard primitive in causal discovery.

In plain words

HSIC maps each variable through a kernel (think: into a rich space of smooth features) and measures the covariance between the two feature sets. It is zero only when the variables are genuinely independent, so it catches arbitrary nonlinear dependence. It is the dependence test that modern causal-discovery algorithms lean on.

import numpy as np
def rbf(v):
    v = v.reshape(-1, 1); d2 = (v - v.T) ** 2
    s = np.sqrt(np.median(d2[d2 > 0]) / 2) + 1e-9
    return np.exp(-d2 / (2 * s ** 2))
def hsic(x, y):
    n = len(x); H = np.eye(n) - 1 / n
    K, L = rbf(x), rbf(y)
    return np.trace(K @ H @ L @ H) / (n - 1) ** 2
print("HSIC =", hsic(x, y))

Gretton, Bousquet, Smola & Scholkopf (2005), ALT; Gretton et al. (2008), NeurIPS (test).

Recent rank correlation that's asymmetric and consistent against any dependence, including noisy functional relationships; cheap, and good at flagging 'Y is a function of X' couplings that symmetric measures blur.

In plain words

Chatterjee's xi (2021) is near zero when two variables are independent and near one when one is a (possibly noisy) function of the other. Unlike Pearson or Spearman it is asymmetric and detects ANY functional shape. It is a few lines of code and scales to large data.

import numpy as np
def xicor(x, y):                          # estimates xi(X -> Y)
    n = len(x)
    order = np.argsort(x)
    r = np.argsort(np.argsort(y[order])) + 1.0       # ranks of y by x-order
    return 1 - 3 * np.abs(np.diff(r)).sum() / (n ** 2 - 1)
print("xi(X->Y) =", xicor(x, y), "  xi(Y->X) =", xicor(y, x))

Chatterjee, S. (2021). A new coefficient of correlation. J. Amer. Statist. Assoc. 116.

Supremum of correlation over all transformations of each variable; the theoretical ceiling on nonlinear association.

In plain words

Ask: what is the best Pearson correlation you could get if you were allowed to reshape each variable through any transformation? That maximum is the Hirschfeld-Gebelein-Renyi (HGR) maximal correlation. The ACE algorithm finds the transformations that achieve it, so you both score the dependence and see the shape that straightens it.

import numpy as np
from ace.ace import ACESolver            # pip install ace

s = ACESolver(); s.specify_data_set([x], y); s.solve()
rho_max = np.corrcoef(s.x_transforms[0], s.y_transform)[0, 1]
print("maximal correlation =", abs(rho_max))

Renyi (1959), Acta Math. Hungar. 10; Breiman & Friedman (1985), JASA 80 (ACE).

Separates the dependence structure from the marginals; tail dependence captures joint extreme behavior invisible to body-of-distribution correlation.

In plain words

A copula is the dependence pattern you get after converting each variable to its rank (uniform) scale, stripping away the individual distributions. That isolates HOW the two move together. Tail dependence is a special slice: the chance both hit extreme values at the same time, which ordinary correlation, dominated by the bulk, barely registers.

import numpy as np
def lower_tail_dep(a, b, q=0.05):
    u = (np.argsort(np.argsort(a)) + 1) / (len(a) + 1)   # pseudo-obs
    v = (np.argsort(np.argsort(b)) + 1) / (len(b) + 1)
    return ((u <= q) & (v <= q)).mean() / q              # ~ lambda_L
print("lower-tail dependence =", lower_tail_dep(stress_a, stress_b))

Joe (1997), Multivariate Models and Dependence Concepts; Nelsen (2006), An Introduction to Copulas.

Scalable nonlinear, near-Pearson cost; practical for wide fleet feature matrices.

In plain words

RDC gets most of the nonlinear-detection power of the heavy methods for almost the price of a correlation. It rank-transforms each variable, projects through a handful of random sine features, then takes the canonical correlation between the two feature sets. Cheap enough to run across thousands of feature pairs.

import numpy as np
from sklearn.cross_decomposition import CCA
def rdc(x, y, k=20, s=1/6):
    def feat(z):
        z = (np.argsort(np.argsort(z)) + 1) / len(z)       # copula transform
        W = np.random.randn(1, k) * s; b = np.random.rand(k) * 2 * np.pi
        return np.sin(z[:, None] * W + b)
    u, v = CCA(1).fit_transform(feat(x), feat(y))
    return abs(np.corrcoef(u.ravel(), v.ravel())[0, 1])
print("RDC =", rdc(x, y))

Lopez-Paz, Hennig & Scholkopf (2013). The randomized dependence coefficient. NeurIPS.

Does A's past improve prediction of B; the linear workhorse for fault-propagation chains.

In plain words

A 'Granger-causes' B if adding A's recent history to a model of B's own history makes B more predictable. It is about predictive precedence, not philosophical cause, but for tracing how a disturbance moves through a system it is the practical first tool. It assumes mostly linear dynamics.

import numpy as np
from statsmodels.tsa.stattools import grangercausalitytests

data = np.column_stack([B, A])            # does A (col 2) cause B (col 1)?
res = grangercausalitytests(data, maxlag=5, verbose=False)
pmin = min(res[l][0]["ssr_ftest"][1] for l in res)
print("min p-value (A -> B):", pmin)

Granger, C. W. J. (1969). Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37.

Model-free, nonlinear directed information flow; the nonlinear generalization of Granger, strong for tracing root cause through coupled subsystems.

In plain words

Transfer entropy measures how many bits of B's future are explained by A's past, beyond what B's own past already explains, with no assumption about linearity. Compute it both ways and the larger direction is the dominant information flow. It is Granger causality freed from the straight-line assumption.