A field guide to computer algebra for engineers

Letters Before Numbers

Symbolic math keeps the letters. Numeric math replaces them with decimals as early as possible. Almost every mistake people make with Wolfram, Mathematica, SymPy and NumPy comes from confusing the two. This is what the split actually is, why physics models want the symbolic side first, how to prove a derivation right before you ship it, which tool does which job, and how both sides are used to root cause a failure and forecast the next one.

Part I

Two kinds of math

Before any tool names, the split that everything else hangs on.

01

The one idea worth keeping

Write down the kinetic energy of a moving mass. There are two completely different things a computer can do with it.

The numeric reading says: tell me the mass and the speed, right now, and I will hand you back a number. Mass is 2.0. Speed is 3.0. Energy is 9.0. Done. The letters are gone the instant you supply values, and what survives is a single decimal.

The symbolic reading says: I will keep the letters. I know that m is a thing, that v is a thing, and that this expression is one half times m times v squared. I do not need to know what they are. I can still differentiate it, rearrange it, solve it for v, or hand you the exact answer with a square root sign still in it.

In plain words

A number is a decision. A symbol is a promise that you have not made the decision yet. Symbolic math is math done before the decision, which is exactly why it can rearrange things: nothing has been thrown away.

That is the whole distinction. Everything below is a consequence of it. The reason it matters in engineering is that a physics model is a statement about relationships, not a statement about one operating point. You want to know that torque scales with current, not merely that torque was 41.2 newton metres yesterday afternoon.

Figure 1

The same expression, read two ways

Pick an expression, then compare the two readings. The numeric column has to commit to values before it can do anything. The symbolic column can answer questions the numeric column cannot even be asked, because it still knows what the parts are. All symbolic results shown were computed with SymPy 1.14 and pasted verbatim.
02

What the computer actually stores

A numeric system stores a block of memory: 64 bits per value, laid out end to end. That is what a NumPy array is.

A symbolic system stores a tree. The expression is broken into operations and operands, and each operation becomes a node with children. One half times m times v squared is not a formula in a string, it is a Mul node whose children are the number one half, the symbol m, and a Pow node holding v and 2.

This sounds like a technicality and it is the single most useful mental model you can have. Once you see the tree, every symbolic operation stops being magic:

Figure 2

Expression tree explorer

Click any node to see the subtree it owns and the derivative rule that applies to it. Press "Differentiate" to watch the chain and product rules walk the tree bottom up. The tree, not the printed formula, is what a computer algebra system holds in memory.
Why this matters for you

If you have ever wondered why a CAS can differentiate anything instantly but chokes on some integrals, this is the answer. Differentiation is a downward walk with a fixed rule per node type. Integration has no such rule, so it becomes a search through a space of candidate answers.

03

The price of numbers

Floating point is not the real number line. It is about sixteen significant decimal digits, and the moment you subtract two nearly equal numbers, most of those digits are noise that was already there. This is called catastrophic cancellation, and it is the most common way a correct formula produces a wrong answer.

The classic demonstration is the expression below. Mathematically it approaches one half as x goes to zero. Numerically, if you type it the obvious way, it collapses to exactly zero.

Figure 3

Where the obvious formula falls apart

naive: (1 - cos x) / x² rewritten: 2 sin²(x/2) / x² exact value, 1/2
Both curves are the same function. The rust curve is the formula as written; below roughly x = 1e-8 the subtraction 1 - cos x has no significant digits left and the result snaps to zero. The teal curve is the algebraically identical rewrite that never subtracts nearly equal numbers. A computer algebra system found the rewrite; floating point could not have. Values computed in IEEE 754 double precision.

Notice what happened. The fix was not a better solver or more precision. The fix was an algebraic identity, applied before any number was ever stored. That is the symbolic layer earning its keep. Goldberg's 1991 survey remains the standard reference on why this happens and what to do about it [1].

A smaller version you have already seen

In floating point, 0.1 + 0.2 gives 0.30000000000000004, and (2**0.5)**2 gives 2.0000000000000004. Symbolically the same two expressions are exactly 3/10 and exactly 2, because a rational stays a rational and a square root stays a square root until you ask for digits.

04

The price of symbols

Symbolic math has its own failure mode, and it is the mirror image. Numbers stay the same size forever: a double is eight bytes whether it holds 3 or 3 billion. Expressions do not. They grow, and they grow fast.

The standard name is expression swell. Take a product of n functions and differentiate it n times. The product rule fires at every node, every time, and the term count explodes.

Figure 4

Expression swell, measured

Term count in the expanded n-th derivative of a product of n unknown functions, computed with SymPy. Five functions differentiated five times already produce 126 terms, and none of them can be simplified away because the functions are unspecified. This is why symbolic pipelines put a simplify step, or a hard stop, between derivation and code generation.

Swell is the reason the honest advice is not "use symbolic math for everything". It is: do the derivation symbolically, then leave. The symbolic layer is a workshop, not a factory floor. Section 28 shows the door you leave through.

~16
significant digits in a float64
8 bytes
size of any float64, always
126
terms from 5 products, 5 derivatives
exact
what a symbol costs you nothing to keep
Part II

What a computer algebra system does

Seven operations cover almost everything you will ever ask one for.

05

The seven moves of a CAS

A computer algebra system, or CAS, is any program that manipulates expression trees rather than numbers. Mathematica is one. SymPy is one. Maple, Maxima and MATLAB's Symbolic Math Toolbox are others. They differ enormously in polish and price, and hardly at all in what they are for.

Strip away the marketing and there are seven things you ask a CAS to do.

Figure 5

The seven moves, with real input and output

Every output shown was produced by running the input through SymPy 1.14 and copying the result. Nothing here is illustrative shorthand. Click through all seven; the last one, code generation, is the move that connects this whole page to a running program.

Notice the pattern. Six of the seven take an expression and return an expression. Only the last one leaves the symbolic world, and it leaves on purpose.

06

How it works underneath

You do not need this section to use the tools, but knowing it explains every performance surprise you will hit.

Figure 6

The four layers inside a CAS

Hover any layer for detail. The bottom two layers are cheap and deterministic. The top two are where the hard theorems live, and where a query can take a second, a minute, or forever.

Canonical form

The first job is deciding when two expressions are the same. x + x and 2*x must become one thing. Systems do this by defining an automatic canonical form: sort the arguments of commutative operations, collect like terms, flatten nested sums. This happens on construction, before you ask for anything, which is why typing x + x in SymPy immediately prints 2*x.

The deep problem lurking here is that for sufficiently rich classes of expressions, deciding whether something equals zero is undecidable. Practical systems use heuristics plus randomized numerical testing, which is why simplify occasionally fails to prove something you can see by eye.

Rewriting and pattern matching

Most of a CAS is rules. If you see sin(a)² + cos(a)², replace it with 1. The engine matches patterns against subtrees and applies rewrites, searching for a smaller result. This is a search over a huge space, which is why simplify is slow and why targeted functions such as trigsimp, collect or cancel are usually faster and more predictable.

Polynomial algebra and Grobner bases

Anything involving systems of polynomial equations rests on Grobner bases, introduced in Bruno Buchberger's 1965 doctoral thesis [2]. A Grobner basis is a rewritten form of a polynomial system that makes questions like "does a solution exist" and "eliminate this variable" mechanical. Solving nonlinear simultaneous equations, eliminating intermediate variables from a kinematic chain, checking observability of a polynomial model: all of it runs on this machinery. Worst-case cost is doubly exponential, which is your warning label.

Integration and the Risch algorithm

Symbolic integration is genuinely different from differentiation. Robert Risch showed in 1969 that for a large class of elementary functions there is a decision procedure: it will either produce an elementary antiderivative or prove that none exists [3]. Full implementations are enormous, so real systems layer heuristics first and fall back to Risch pieces. That is why some integrals return instantly, some grind, and some come back unevaluated.

Rule of thumb

Differentiate freely, integrate hopefully. If you need an integral in production code, get it once in the CAS, check it by differentiating the answer back, then hard code it.

07

When no formula exists

A CAS is not a wish granting machine. Some things have no closed form and no amount of software will change that.

Figure 7

Three requests, three different kinds of answer

Exact, named, or refused. Learning to read which of the three you got is most of the skill of using a CAS well.

This is not a defeat. Knowing that no elementary antiderivative exists is itself a result, and it tells you immediately to reach for quadrature instead of hunting for a formula that was never there.

Part III

Physics based modeling, in symbols

Where this stops being a math curiosity and starts being how models get built.

08

The physics pipeline

"Physics based" modeling, sometimes called white box modeling, means the model comes from a conservation law or a constitutive relation rather than from fitting a curve to data. Newton's second law, an energy balance, Kirchhoff's laws, Maxwell's equations, Fourier's law of conduction. You start from the statement of the physics and derive your way to something you can compute.

That derivation is the part people still do by hand on paper, and it is the part that is easiest to get wrong. A missing factor of two in a Park transform, a dropped cross term in a Jacobian, a sign flipped in a coordinate rotation: none of these announce themselves. They show up three weeks later as an estimator that will not converge.

The symbolic layer is where that derivation belongs.

Figure 8

From physical law to running code

Hover a stage for what happens there and which tool does it. The single most important feature of this pipeline is the vertical line: everything left of it keeps letters, everything right of it holds numbers, and the crossing happens exactly once, deliberately, by generating code.
In plain words

Derive with letters, run with numbers, and cross the border on purpose rather than by accident. Most slow, fragile scientific code crosses that border in the first five minutes and then spends the rest of its life paying for it.

09

Worked example: the pendulum

The smallest honest example of a symbolic derivation. A mass on a rigid massless rod. We will not write Newton's laws with free body diagrams and force components. We will write down two energies and let the algebra produce the equation of motion.

This is the Lagrangian recipe, and it is the standard way multibody and driveline models get derived. Write the kinetic energy T, write the potential energy V, form L = T - V, then apply the Euler-Lagrange equation. What comes out the other side is the equation of motion, with every cross term and coupling included automatically.

Figure 9

Euler-Lagrange, one step at a time

Step through the derivation. Every intermediate expression is genuine SymPy output, in SymPy's own printed form, not a cleaned up transcription. The whole derivation is nine lines of code.

The last line is the thing you wanted, and you never touched a force diagram. For a pendulum that is a parlour trick. For a five link suspension or a planetary gearset it is the difference between a model that exists and one that does not.

What the small angle approximation actually costs

The derived equation contains sin(theta), which is why the pendulum is nonlinear. Almost every control textbook then replaces sin(theta) with theta, using the series the CAS will hand you: sin(x) = x - x³/6 + x⁵/120 - .... That substitution is what makes the pendulum a linear oscillator with a constant natural frequency.

The series is also how you quantify the error, instead of hoping for the best.

Figure 10

Nonlinear pendulum against its linearized twin

nonlinear, sin θ linearized, θ
Both pendulums start from the same angle and are integrated with the same fourth order Runge-Kutta step. At small angles they are indistinguishable. Past about 30 degrees the linearized model runs visibly fast, because it has thrown away the term that slows a real pendulum down at large amplitude. The period error shown is measured from the simulation, against the exact period expressed through the complete elliptic integral of the first kind.
10

Worked example: motor torque

A permanent magnet synchronous machine, written in rotor coordinates. This is the derivation that R. H. Park's 1929 two-reaction theory made possible [4], and it is a good example because the useful result is a formula, not a number.

In the rotor frame the flux linkages are the magnet flux plus the current times inductance in each axis. Torque is the cross product of flux and current, scaled. Written symbolically that is three lines, and expanding them gives the standard result that every field oriented controller is built on.

SymPy inputlam_d = lam_m + L_d*i_d
lam_q = L_q*i_q
Te = Rational(3,2)*p*(lam_d*i_q - lam_q*i_d)
collect(expand(Te), i_q)
Outputi_q*(3*L_d*i_d*p/2 - 3*L_q*i_d*p/2 + 3*lambda_m*p/2)

Read that back and it is the two term torque equation: a magnet term proportional to the magnet flux and the q-axis current, plus a reluctance term proportional to the difference between the two inductances and the product of both currents. A surface magnet machine has the two inductances nearly equal, so the second term vanishes and torque is simply proportional to q-axis current. An interior magnet machine has a real saliency, so the second term is worth having and you must go looking for it.

Now push the symbolic step one stage further. Write the currents in polar form with a total magnitude and a current angle, differentiate the torque with respect to that angle, and solve for zero. The result is the maximum torque per ampere angle, in closed form.

Figure 11

Torque against current angle, with the derived MTPA point

total torque magnet term reluctance term
Four poles assumed. The dotted vertical line is not found by searching the curve; it is placed by evaluating the closed form expression that SymPy returned for the stationary point of torque with respect to current angle. With the default values it lands at 24.356 degrees, which agrees with a brute force search over 200,001 angles to five decimal places. Set Ld equal to Lq and the reluctance term flattens to zero and the optimum snaps to zero degrees, which is the surface magnet case.
The point of the example

A numerical optimizer could have found that angle too, by evaluating torque at many angles and picking the best. But it would have to do that again for every operating point, forever, and it would give you a number rather than an understanding. The symbolic derivation gives you a formula you can put in a lookup free controller, differentiate again for sensitivity, and inspect to see that the optimum depends on saliency and current but not on pole count.

11

Worked example: heat, and how state space appears

The third pattern is the one you will use most often: take a nonlinear physical model, and get the linear state space matrices that an observer or a controller needs. Symbolically, that is one function call.

Consider a two node thermal model. A heat source dumps power into node one, node one conducts to node two, node two conducts to ambient. Two capacitances, two conductances. Writing the energy balance for each node gives two first order differential equations, and the Jacobian of that pair with respect to the states is the A matrix.

SymPy inputf1 = (q - k_12*(T1 - T2)) / C_1
f2 = (k_12*(T1 - T2) - k_2a*(T2 - T_a)) / C_2
F = Matrix([f1, f2])
A = F.jacobian(Matrix([T1, T2]))
B = F.jacobian(Matrix([q]))
OutputA = Matrix([[-k_12/C_1, k_12/C_1],
          [ k_12/C_2, (-k_12 - k_2a)/C_2]])
B = Matrix([[1/C_1], [0]])

Those matrices arrived with the physical parameters still in them. That is the whole prize. You can read the structure directly: the coupling term appears twice with opposite roles, the input only enters the first state, and every entry is a conductance over a capacitance, which has units of one over seconds. Substituting numbers in gives poles, time constants, and a discretized model for an estimator.

Figure 12

Same symbolic model, live numbers

node 1, core node 2, case
Step response to 200 W applied at the core, from ambient. The matrix above the plot is the symbolic A matrix with your slider values substituted in; the eigenvalues and time constants are computed from it. Drag the core capacitance down and watch the fast time constant shrink while the slow one barely moves: the two constants belong to different physical mechanisms, and the symbolic form is what lets you see which is which.
12

Why exact derivatives pay off

The single most common reason an engineer needs symbolic math is not integration or simplification. It is derivatives, and specifically Jacobians.

You can always approximate a derivative with finite differences. The problem is that finite differences force you to trade two errors against each other and there is no step size that removes both.

Figure 13

The finite difference trap

forward difference central difference symbolic derivative
Relative error in the first derivative of exp(sin 3x)/(1+x²) at x = 0.7, against step size, in double precision. Large steps carry truncation error; small steps carry cancellation error, the same effect as Figure 3. The best a forward difference ever manages here is about one part in 10⁷, near a step size of the square root of machine epsilon. The symbolic derivative, evaluated at the same point, sits at the bottom of the plot at machine precision for every step size, because it never took a step at all.
The honest caveat

Symbolic differentiation is not the only exact option. Automatic differentiation, the technique behind JAX and PyTorch, also produces derivatives correct to machine precision, without ever building the symbolic expression [5]. The difference is that AD gives you the derivative's value at a point, while symbolic gives you its formula. If you want to read the formula, prove something about it, or ship it as fixed C code with no runtime dependency, you want symbolic. If you just want the number, fast, on a million points, AD is usually the better tool.

Part IV

Trusting the derivation

A symbolic result is not correct because a computer produced it. Here is how you find out.

13

Five checks that catch nearly everything

The failure mode of symbolic work is not a crash. It is a clean, plausible, well typeset expression that is wrong, because the physics you fed in was wrong. The algebra was flawless. The Lagrangian had a sign error.

These five checks are cheap, they are mostly automatable, and between them they catch the overwhelming majority of real errors.

  1. The degenerate case. Set a parameter to the value where you already know the answer. Saliency to zero, and the reluctance torque term must vanish. Damping to zero, and the response must not decay. Coupling to zero, and the two subsystems must separate. This is the single highest yield check because you always have at least one such case.
  2. Equilibrium and its sign. Find where the derivative is zero, then differentiate once more. The sign of that second derivative tells you whether the equilibrium is stable, and you know from physics which it should be. A hanging pendulum is stable. If your equation says otherwise, stop.
  3. A conserved quantity. If the system is conservative, integrate it and watch the energy. With a correct model and a decent integrator the drift is at machine level. With a wrong model it is not.
  4. Dimensions. Every term in a sum must carry the same units. This is entirely mechanical and a computer can do it for you, which is the subject of the next section.
  5. A limit or asymptote. What happens as a parameter goes to zero or infinity? A CAS will take that limit exactly. The answer usually has an obvious physical reading, and if it does not, something is wrong.
And for integrals specifically

Differentiate the answer and simplify the difference against the original integrand. If it is not zero, you have your answer about the answer. This takes one line and it is the only integral check you need.

14

What a wrong derivation looks like

Rather than describe this, here it is done. Four realistic errors were injected into the derivations from Part III, and each of the five checks was run against each. The results below are measured, not asserted.

Figure 14

Error injection against the five checks

Click an error to see the corrupted expression, the check results, and the measured evidence. Every number was produced by actually running the broken model. Note the pattern in the last column: no single check catches everything, and the cheapest check is rarely the one that fires.

Two things are worth pulling out of that.

First, the sign flipped pendulum is caught instantly by the equilibrium check: the stiffness comes out as plus g over L instead of minus g over L, which says the pendulum is stable pointing upward. You do not need a simulation to see that, and you do not need to be clever. You need to have asked.

Second, the sign flipped motor torque is not caught by peak torque, which is identical at 123.75 newton metres either way. It is caught only by looking at where the peak is: the correct model puts it at plus 24.356 degrees, the broken one at minus 24.356 degrees. One asks for negative d-axis current, which is what a real interior magnet machine wants. The other asks for positive d-axis current, which magnetizes the machine further and is physically backwards. A scalar check would have missed this entirely.

15

Units as a type system

Dimensional analysis is the check engineers were taught first and automate last. There is no reason for that. SymPy carries a units module, and once quantities carry their dimensions a whole class of errors becomes impossible rather than merely unlikely.

Inputfrom sympy.physics.units import watt, joule, kelvin
from sympy.physics.units.systems.si import SI

k12 = 6*watt/kelvin ; C1 = 800*joule/kelvin ; q = 200*watt
SI.get_dimensional_expr((q - k12*(T1 - T2)) / C1)
Outputpower*temperature/energy

# which reduces to 1/time, as a rate of change of temperature must

Now mistype the capacitance as joules rather than joules per kelvin, the exact slip a tired person makes at five o'clock:

InputSI.get_dimensional_expr((q - k12*(T1 - T2)) / (800*joule))
Outputpower/energy

# temperature has silently vanished. The expression still evaluates.
# It is still wrong.

And an outright category error, adding a power to a heat capacity, raises rather than returning nonsense:

Inputfrom sympy.physics.units.util import check_dimensions
check_dimensions(q + C1)
OutputValueError: addends have incompatible dimensions
Figure 15

Dimensional consistency, term by term

Each row is one term of a candidate equation, with its dimensions resolved to the four SI base dimensions used here. A sum is legal only when every row shows the same exponent vector. Toggle the injected mistake to watch a single exponent change and the whole equation become meaningless while remaining perfectly computable.
16

How many knobs are really there

Dimensions do more than catch mistakes. They tell you how many independent parameters your model actually has, which is almost always fewer than the number you wrote down.

The Buckingham Pi theorem, from Edgar Buckingham's 1914 paper [22], says it precisely: if a physical relationship involves n variables built from r independent dimensions, it can be rewritten in terms of exactly n minus r dimensionless groups. Mechanically, you write the exponents of each variable as a column, and the null space of that matrix is your set of groups.

Applied to the two node thermal model from Section 11: seven quantities (heat input, two conductances, two capacitances, time, temperature rise), a dimensional matrix of rank three, and therefore four dimensionless groups. Not seven knobs. Four.

Figure 16

Seven variables, four real parameters

Toggle variables in and out of the problem and watch the count of independent dimensionless groups follow. The groups shown are the actual null space basis, computed in the page by the same rational elimination a CAS uses, and matching SymPy's answer for the full set. The four for the full model come out as the conductance ratio, the capacity ratio, time measured in units of the core time constant, and temperature measured in units of the driving power over the coupling conductance.
Why an engineer should care

A parameter sweep over seven variables at ten points each is ten million runs. A sweep over four is ten thousand. The reduction is not an approximation, it is exact: the extra three dimensions were never independent information. Non-dimensionalizing before you sweep, calibrate, or train anything is one of the highest leverage symbolic steps available, and almost nobody does it.

17

The simplest form is not the most stable form

This one contradicts an instinct, so it is worth stating flatly. simplify optimizes for a small expression tree. It does not know anything about floating point. The shortest form and the numerically safest form are different objects, and sometimes they are opposites.

The quadratic formula is the canonical case. Both roots of a monic quadratic come out of the same expression with a plus or minus. When the linear coefficient is much larger than the constant, one of those two roots is computed by subtracting two nearly equal numbers, and Section 3 already told you what happens next.

The fix is Vieta's relation: the product of the roots equals the constant term. Compute the large root by the formula that does not cancel, then get the small root by division.

Figure 17

Two algebraically identical formulas, one of which works

textbook formula Vieta rearrangement
Relative error in the smaller root of x² + bx + 1, double precision, measured against the rearranged formula, which SymPy confirms is correct to machine precision across this whole range. At b = 10⁸ the textbook formula is wrong by 25 percent. The rearranged version is correct to machine precision throughout. No solver, no extra precision, no iteration: a different but equal algebraic form.

The general statement is that an expression has a condition number, a measure of how much a small relative change in the inputs can amplify into the output, and algebraically equivalent forms can have wildly different ones. A CAS will happily hand you either. Choosing between them is engineering judgement that the tool cannot make for you.

Three places this bites in practice

Subtracting two nearby temperatures or positions to get a small difference. Computing 1 - exp(-x) for small x, where expm1 exists precisely to avoid the cancellation. And inverting a matrix symbolically, then substituting numbers, when solving the linear system numerically would have been both faster and better conditioned.

Part V

Control and estimation

The place where keeping the letters pays for itself several times over.

18

Controllability and observability, in symbols

The rank tests for controllability and observability are usually taught as numerical procedures on numerical matrices. Done symbolically they become far more useful, because instead of a yes or a no you get the condition under which the answer changes.

Take the A and B matrices derived in Section 11 and build the controllability matrix. Its determinant comes out in one line:

InputCtrb = Matrix.hstack(B, A*B)
simplify(Ctrb.det())
Outputk_12/(C_1**2*C_2)

Read that back and it says everything. The system is controllable for every physically meaningful parameter value, and it loses controllability exactly when the coupling conductance goes to zero, which is the physical statement that if the core is thermally disconnected from the case you cannot influence the case by heating the core. That is not a fact you would have learned from a numerical rank test at one operating point. It came out of the algebra.

Observability behaves the same way, and the same conductance is the culprit whichever node you measure:

Sensordet of the observability matrixReading
case temperature only-k_12 / C_2observable unless the nodes are decoupled
core temperature onlyk_12 / C_1same condition, from the other side
Figure 18

Watching a rank condition degrade

Rank is a yes or no answer and it is the wrong question in practice. What matters is how close to singular you are, which the smallest singular value of the controllability matrix measures. Drag the conductance toward zero. The determinant and the smallest singular value fall together while the condition number climbs, and all three say the same thing with far more nuance than a rank test. The symbolic determinant told you which parameter to put on this slider.
19

From state space to a transfer function

The map from a state space model to a transfer function is a matrix inverse, a matrix product and a simplification, all of which a CAS does exactly. What arrives is a formula in your physical parameters rather than a table of coefficients.

InputCm = Matrix([[0, 1]]) # measure the case
G = simplify(Cm * (s*eye(2) - A).inv() * B)[0,0]
cancel(together(G))
Output                 k_12
-------------------------------------------------
C_1*C_2*s**2 + (C_1*k_12 + C_1*k_2a + C_2*k_12)*s + k_12*k_2a

Three readings fall straight out of that expression, and none of them would be visible in a numerical Bode plot.

20

Structural identifiability

Here is the question that decides whether a calibration campaign is worth running at all: given perfect, noise free data from the sensors you actually have, can the parameters be recovered uniquely?

Notice this is not a question about noise, sample rates or excitation. It is a question about the structure of the model, and it can be answered before a single measurement is taken. The standard treatment casts it as a differential algebra problem [23], and the machinery underneath is the same polynomial elimination from Section 6.

For the two node model measured at the case, the transfer function has exactly three independent coefficients. There are four unknown parameters. Three equations, four unknowns, and the conclusion is immediate: the parameters are not individually identifiable. Ask SymPy to solve anyway and it tells you so by leaving one free:

Inputsolve([Eq(a1, alpha1), Eq(a0, alpha0), Eq(b0, beta0)],
      [C_1, C_2, k_12, k_2a], dict=True)
OutputC_1 = beta0*(alpha1 - C_2*beta0)/(alpha0 + beta0**2)
k_12 = beta0
k_2a = alpha0/beta0

# C_2 never gets solved. It is free.

This sounds abstract until you look at what it costs. Below are three parameter sets, all physically plausible, all producing case temperature histories that agree to within a ten trillionth of a degree. They disagree about the core temperature by seventeen degrees.

Figure 19

Identical measurements, different unmeasured truth

case, measured core, inferred nominal parameter set
Slide the free parameter. The other three parameters are recomputed from the closed form above so that the transfer function stays exactly the same, and the teal curve therefore does not move at all: every setting reproduces the measured case temperature to within 5×10⁻¹³ K. The rust curve is the core temperature the same model predicts, and it moves by tens of degrees. If your thermal observer estimates a junction temperature from a case sensor, this figure is the risk.
What to do about it

Three options, in order of preference. Add a sensor, which changes the structure and can make the model identifiable. Fix a parameter from an independent source, such as computing a capacitance from mass and specific heat rather than fitting it. Or reparameterize in terms of the combinations that are identifiable and stop pretending you know the rest. What you must not do is fit all four, get a good residual, and believe the individual numbers.

21

Discretization, and a warning about the matrix exponential

A continuous model has to become a difference equation before it runs in a controller. The exact answer is the matrix exponential of A times the sample period, and a CAS will compute it symbolically. It is also a good demonstration of why you should not want it to.

7
operations in the symbolic A matrix
1,410
operations in the symbolic exp(A·T)
201×
expression swell, for a 2 by 2

For a single first order state the closed form is exp(-T/tau) and you should absolutely keep it symbolic. For two states it already runs to hundreds of terms full of atan2 and complex parts. The correct move is to keep A symbolic, substitute numbers, and exponentiate numerically. Deriving symbolically does not mean staying symbolic to the last possible instant; it means staying symbolic while the symbols are still telling you something.

The practical question is which approximation to use, and the honest answer depends entirely on the sample period relative to the fast time constant.

Figure 20

Three discretizations, and where two of them fail

forward Euler Tustin, bilinear exact, matrix exponential
Magnitude of the fast discrete pole against sample period, for the thermal model of Section 11. The exact pole is the exponential of the continuous pole and stays inside the unit circle for every sample period, always. Tustin maps the left half plane into the unit disc and is therefore also unconditionally stable, at the price of frequency warping. Forward Euler crosses the unit circle: past a sample period of about 220 seconds this model goes unstable in simulation while the physical system it represents obviously does not. Plotted for the fast pole, which is the one that sets the limit.
22

Differential algebraic equations and index reduction

Everything so far has assumed you can write the model as derivatives on the left and functions on the right. Physical models built from connected components usually cannot. Connect two components and you get constraints, not derivatives: currents at a node sum to zero, two shafts share an angle, a bearing holds a distance fixed. What you have is a differential algebraic equation, and handing it to an ODE integrator does not work.

Write the pendulum in Cartesian coordinates and you have the smallest honest example. Two equations of motion with an unknown rod tension, plus one constraint that the length is fixed:

The model as a physicist writes itm*x'' = -lam*x
m*y'' = -lam*y - m*g
x**2 + y**2 - L**2 = 0
The problem# the constraint contains no derivative of lam.
# there is no equation for how the tension evolves.
# an ODE solver has nothing to integrate.

The fix is mechanical: differentiate the constraint with respect to time until a derivative of the missing variable appears. The number of differentiations needed is the differentiation index, and finding it automatically is what Pantelides' algorithm does [24], which is the machinery running silently inside every Modelica tool and inside Simscape.

Figure 21

Index reduction, one differentiation at a time

Step down the ladder. Each level is the previous constraint differentiated once, and the level names are the standard ones from multibody dynamics: position, velocity, acceleration. Two differentiations are needed, so the model is index 3. The tension that falls out at the bottom is the classic result, and it is worth reading physically.

The answer at the bottom of that ladder is worth a moment. The tension comes out as the mass times the speed squared minus gravity resolved along the rod, all over the length squared. That is centripetal force minus the weight component, exactly as it should be, and it was never written down by anyone. It was produced by differentiating a geometric constraint twice and substituting.

Why this matters even if you never write a DAE

When a Simscape or Modelica model runs slowly, fails to initialize, or drifts off its constraint surface, index reduction is usually the reason. High index systems need consistent initial conditions and they suffer constraint drift that an ordinary integrator will not correct. Knowing that a symbolic differentiation step is happening under the block diagram is the difference between debugging the model and reinstalling the software.

Part VI

Shipping it

The last mile, from a correct expression to code on a target.

23

cse, Horner, and counting operations

An expression that is correct can still be a bad program. Two transformations do most of the work of fixing that, and both are one function call.

Horner form

A polynomial written out as a sum of powers evaluates each power separately. Nested multiplication evaluates the same polynomial with only multiplies and adds, and no exponentiation at all. The operation count barely changes. The runtime does not.

InputP = 3*x**5 - 7*x**4 + 2*x**3 + 9*x**2 - 4*x + 11
horner(P)
Outputx*(x*(x*(x*(3*x - 7) + 2) + 9) - 4) + 11

Common subexpression elimination

A Jacobian repeats the same subtrees over and over, because the same inner functions appear in every entry. cse finds them and hoists them into temporaries. On a three by two Jacobian of a fairly ordinary nonlinear model, it removed two thirds of the arithmetic.

Figure 22

What the two transformations actually buy

Operation counts from SymPy's count_ops, wall clock from evaluating the generated functions over two million points on the machine that built this page. The Horner speedup is large because raising to an integer power calls a general pow routine, while the nested form is nothing but multiply and add. The cse speedup in NumPy is modest because array evaluation is limited by memory bandwidth rather than arithmetic; the same transformation on scalar embedded C, where every operation is real work, buys far more.
Read the caption carefully

The op count fell by 66 percent and the NumPy runtime fell by 10 percent. Both numbers are true and they are measuring different machines. Optimizing arithmetic is worth a great deal on a microcontroller and comparatively little inside a vectorized array pipeline, where the bottleneck is moving data. Know which target you are on before you spend effort here.

24

The road to embedded C

The full path from a physical law to a function running on an automotive microcontroller has five steps, and SymPy covers three of them.

  1. Derive and verify using everything in Parts III and IV. Nothing goes further until the degenerate cases pass.
  2. Reduce. cse to hoist repeats, horner on polynomials, and collect to group by the variable that changes fastest. Look at count_ops before and after.
  3. Generate. codegen emits a compilable C function and its header. ccode emits a single expression. Both accept a settings object that controls how constants and integer powers are printed.
  4. Check the numerics on the target's arithmetic. This step is yours, not the tool's. Single precision on an embedded target has about seven significant digits, not sixteen, so every conditioning concern from Section 17 gets worse by nine orders of magnitude. Re-run the cancellation checks in float32 before you trust anything.
  5. Wire the generated file into the build as an artifact, never as source. The derivation script is the source. If a parameter definition changes, you regenerate and the diff is reviewable. Patching generated code by hand is how a model and its implementation quietly diverge.
Inputexpr = J[0,0]
repl, reduced = cse(expr)
for sym, sub in repl: print('double', sym, '=', ccode(sub) + ';')
print('result =', ccode(reduced[0]) + ';')
Generated Cdouble x0 = b*y;
double x1 = a*x;
result = -a*exp(-x1)*sin(x0) + (1.0/2.0)*a/sqrt(x0 + x1);

Note what SymPy did with the one half: it printed (1.0/2.0) rather than 0.5. That is a deliberate choice about how the C compiler will read the constant, and it is the kind of detail that separates a code generator from a pretty printer.

The single most useful habit

Generate the C, then generate a NumPy version of the same expression, then assert they agree to tolerance on a few thousand random inputs across the operating envelope. It takes twenty minutes to set up and it has caught, in the experience of everyone who has ever done it, at least one thing.

Part VII

The four names

Two of these compete with each other. One of them is not in the same category at all.

25

Wolfram Language and Mathematica

The first thing to untangle is that "Wolfram" and "Mathematica" are not two products, and people use the words interchangeably in a way that hides the structure.

The language itself is unusual and worth understanding on its own terms: everything is an expression, evaluation is term rewriting, and pattern matching is the primary control structure. A definition like f[x_] := x^2 is not a function in the C sense, it is a rewrite rule that fires whenever the pattern matches. That design is exactly why the system is so strong at symbolic work, and exactly why its performance is hard to predict.

Where it genuinely leads: symbolic integration and differential equations, special functions, exact and arbitrary precision arithmetic with real numerical rigour, and simplification. When SymPy returns an expression unsimplified and Mathematica returns three terms, this is usually why. Version 15 shipped in June 2026 with a built in AI assistant, an expanded time series framework, and additions on the algebra side including Grassmann and Clifford algebras [6].

The costs are real: it is proprietary, licensing is a recurring negotiation, and the language does not embed comfortably in an existing Python or C toolchain. In a fleet analytics or embedded workflow, that last point usually decides it.

26

SymPy

SymPy is a computer algebra system written in pure Python, first released in 2007 and described in a 2017 paper in PeerJ Computer Science [7]. It is free, BSD licensed, and has one design decision that dominates everything else: it is a library, not an application. There is no notebook, no separate kernel, no license server. You import sympy in the same file where you import numpy.

That single property is why it wins in an engineering pipeline even though it loses to Mathematica in raw symbolic strength. The symbolic derivation happens in the same process, same repository, same version control, and same test suite as the numerical code that consumes it.

# derive, simplify, and emit runnable code, in one file
import sympy as sp
import numpy as np

x, y = sp.symbols('x y', real=True)
f = sp.Matrix([sp.sin(x)*y**2, sp.exp(-x)+y])

J = f.jacobian([x, y])            # exact 2x2 Jacobian, still symbolic
J_fast = sp.lambdify((x, y), J, 'numpy')   # leave the symbolic world

J_fast(0.3, 1.2)                      # plain numpy array, microseconds

Practical notes that will save you an afternoon:

Inputcodegen(('field', sin(x)**2 + exp(-x**2)), 'C99')
Generated Cdouble field(double x) {
   double field_result;
   field_result = pow(sin(x), 2) + exp(-pow(x, 2));
   return field_result;
}

The honest limits: SymPy is slower than a compiled CAS, its simplify gives up earlier, and hard integrals and differential equations are noticeably weaker than Mathematica's. For the derivations engineers actually do, Jacobians, coordinate transforms, series expansions, solving for a variable, none of that is the binding constraint.

27

NumPy

NumPy does not belong in the same comparison. It has no symbols, no expression trees, and no algebra. Putting it in a list with Mathematica and SymPy is like listing a lathe alongside two CAD packages. It is the thing that runs afterward.

What NumPy provides is one data structure, the n-dimensional array, and a large set of operations that act on the whole array at once [8]. Three properties do all the work:

Figure 23

Broadcasting, visualized

Shapes are compared from the trailing axis backward. A dimension of size 1 is stretched to match; nothing is actually copied, the same memory is read repeatedly. This is how a symbolic model that was lambdified into a scalar function evaluates over a whole parameter grid in one call.

SciPy sits directly on top and supplies the things a physics pipeline needs next: integrators, optimizers, linear algebra beyond the basics, signal processing, and interpolation [9]. In practice "NumPy" as a shorthand usually means the NumPy and SciPy pair.

28

The handoff: lambdify

Here is the mistake that motivates this section. Someone builds a beautiful symbolic model, then evaluates it by calling .subs() in a loop over their data. It works. It is also about fifty thousand times slower than it needs to be, and they conclude that symbolic math is impractical.

subs rebuilds an expression tree per call. That is the wrong tool for evaluation. lambdify compiles the tree once into a plain Python function whose body is ordinary arithmetic, closed over NumPy's array operations. After that there is no symbolic machinery in the loop at all.

What lambdify buildsf = sp.lambdify(x, sp.sin(x)**2 + sp.exp(-x**2), 'numpy')
The generated function bodydef _lambdifygenerated(x):
    return sin(x)**2 + exp(-x**2)

That is all it is. No tree, no dispatch, no rewriting. The names sin and exp are bound to NumPy's versions, so passing an array evaluates the whole array in compiled code.

Figure 24

Four ways to evaluate the same expression

Time per evaluated point for sin²(x) + exp(-x²), logarithmic scale, measured on the machine that generated this page. Note the axis: each gridline is a factor of ten. The gap between staying symbolic and crossing over to arrays is four and a half orders of magnitude, and it costs one line of code to cross.
Two things to get right

Pass 'numpy' as the modules argument if you want array support; the default may pick the standard library math, which will reject arrays. And run cse on large expressions first, otherwise a swollen tree becomes a swollen, slow function.

29

Four tools side by side

ToolCategoryWhat it is best atCost and fit
Wolfram LanguageSymbolic and numeric engine Hard integrals, PDEs, special functions, exact and arbitrary precision arithmetic, simplification that actually finds the compact form Commercial. Its own ecosystem. Free Wolfram Engine for developers; callable from Python but not native to it
MathematicaNotebook application on that engine Interactive exploration, publication quality typeset math, curated data, teaching Commercial licence. The interface most people mean by "Wolfram". Version 15 released June 2026
SymPySymbolic library, pure Python Derivations inside a Python pipeline: Jacobians, coordinate transforms, series, solving for a variable, and code generation to C, Fortran or MATLAB Free, BSD. Weaker simplification and integration than Mathematica. Wins on integration with everything else you already run
NumPyNumeric array library Running the finished equations fast on real data. Contiguous memory, vectorized operations, broadcasting Free, BSD. Not a competitor to the other three; it is where their output goes to be executed

The relationship in one line: Mathematica and SymPy are alternatives to each other, and NumPy is what either of them hands off to. Wolfram Language is the engine Mathematica is built on, and it happens to include a strong numeric side of its own, which is why it can be a whole environment rather than one stage of a pipeline.

Figure 25

Which one do you actually need

Answer the questions in order. There is no wrong branch here, only branches that cost you more than they need to.
Part VIII

The neighborhood

Four names are the common ones. These are the tools that solve the problems the four leave over.

30

Beyond the four

Once you understand the symbolic and numeric split, a whole layer of tools stops looking like a random list and starts looking like answers to specific gaps.

ToolWhich gap it fillsReach for it when
CasADi Symbolic expressions built specifically to be differentiated and compiled, with exact derivatives of any order and direct hooks into nonlinear solvers [10] You are writing an MPC or NMPC controller, or any optimization where the solver needs derivatives of your physics at every iteration
Modelica Acausal physical modeling. You declare components and connections; the tool does the symbolic index reduction of Section 22 and equation sorting to produce something solvable [11] You are building a plant model from physical components across domains, and you do not want to hand derive the causality
Simscape The same acausal idea inside Simulink, with the symbolic manipulation hidden under the block diagram The rest of your team already lives in Simulink and the model must sit in an existing controls workflow
Symbolic Math Toolbox A CAS inside MATLAB, sharing MATLAB's workspace and code generation You need the derivation in the same language as everything else you ship, and that language is MATLAB
JAX Not symbolic. Traces numeric code and differentiates it exactly, then compiles it with XLA [12] You want machine precision derivatives at speed but do not need to read the formula
ModelingToolkit.jl Symbolic model building in Julia that applies structural simplification and then compiles to fast numeric code You want the Modelica idea in a general purpose language with the numerics in the same process
StructuralIdentifiability.jl Automates Section 20: decides globally, by differential elimination, which parameters a given output set can recover [23] Before any calibration campaign on a model with more than three parameters
PySR Symbolic regression at production quality: searches expression trees for a formula that fits, returning a whole accuracy against complexity front [25] You have data and suspect there is a compact law in it that you cannot guess
Maxima Free descendant of Macsyma, the original large CAS. Still strong, still maintained You want a full CAS with no licence and do not need Python integration
SageMath A Python front end that unifies many open source math systems, SymPy and Maxima among them The work is closer to mathematics than to engineering

CasADi deserves a second mention because it sits precisely on the seam this page is about. Its expressions are symbolic enough to be differentiated exactly, and restricted enough to be compiled to fast C. It gives up general simplification and integration, which it does not need, in exchange for derivatives and speed, which it does.

Part IX

Learning the equations

Four ways of putting data and symbols on the same side of the table. All four run in this section.

31

SINDy, run for real

Everything so far started from a known law. Sparse identification of nonlinear dynamics, or SINDy, starts from measurements and produces the law [14]. The premise is simple and usually true: most physical systems have sparse dynamics in a sensible basis, meaning that of the hundreds of candidate terms you might write down, only a handful actually appear.

The recipe is three steps. Build a library of candidate terms evaluated on your data. Estimate the derivatives. Then solve for the coefficients with a regression that is pushed toward zeros, typically by least squares followed by thresholding, repeated.

Below is a real run against the Van der Pol oscillator: eight trajectories, ten candidate terms up to cubic, derivatives from finite differences on the states rather than from the true vector field.

Figure 26

The sparsity threshold is the whole method

Coefficients from a real sequentially thresholded least squares run, recomputed as you move the threshold. On clean data the correct three term equation is recovered over a wide range of thresholds and the coefficients land on 1.2000 exactly. Switch to the noisy data and the safe range collapses: too low a threshold keeps spurious terms, too high a threshold starts deleting real ones. That window is the whole practical difficulty of the method.

What you get out is an equation, not a black box. That is the entire point, and it is why SINDy sits in a diagnostics workflow in a way that a neural network does not: you can read the result, check it against physics, and reject it when the terms are nonsense.

The three ways this fails in the field

Derivative estimation from noisy measurements, which is the dominant error source and the reason smoothed or weak formulations exist. A library that does not contain the true terms, in which case the method confidently returns the best wrong answer available. And unmeasured states, because SINDy identifies the dynamics of what you give it, and a hidden state simply is not in the equation.

32

Koopman: making a nonlinear system linear

This idea is nearly a century old, from Bernard Koopman's 1931 paper, and it is one of the strangest and most useful things in dynamics [26]. It says: stop tracking the state, and track functions of the state instead. In that space, nonlinear dynamics become linear. Exactly linear, not approximately.

The catch is that the space of functions is infinite dimensional, so you have traded a hard finite problem for an easy infinite one. The practical question is whether some finite set of observables closes, meaning the operator maps that set into itself.

Sometimes it does exactly. Take a system with a slow manifold, the standard example in the field. It is genuinely nonlinear. Now watch what happens if you add the square of the first state as a third observable.

The nonlinear systemx1' = mu*x1
x2' = lam*(x2 - x1**2)
Symbolically, on the observables [x1, x2, x1**2]d/dt [ x1 ] [ mu 0 0 ] [ x1 ]
     [ x2 ] = [ 0 lam -lam ] [ x2 ]
     [x1²] [ 0 0 2mu ] [x1²]

# a constant matrix. the nonlinearity is gone, not linearized.

That matrix was not guessed. It comes from differentiating each observable along the flow and expressing the result back in the same three observables, which is a symbolic operation and takes four lines of SymPy. The third row is the interesting one: the derivative of x1 squared is twice x1 x1 prime, which is twice mu times x1 squared, so the observable closes on itself.

Extended dynamic mode decomposition, or EDMD, is the data driven version [27]: pick a dictionary of observables, lift your snapshot data through it, and least squares fit the matrix that advances one lifted snapshot to the next. Run against the system above, it recovers the exact operator.

Figure 27

Lifting, and what it buys

true trajectory lifted linear model, EDMD unlifted linear model, DMD
Both linear models are fitted from the same 6,000 snapshot pairs by the same least squares. The only difference is that one of them was handed a third observable. Adding it takes the one step prediction error from 1.7×10⁻² down to 6.6×10⁻¹⁶, which is machine precision, and recovers the continuous eigenvalues as exactly the set the symbolic derivation predicted.

Two things follow that are worth carrying away.

First, the choice of observables is a modeling decision, not a hyperparameter, and physics is what tells you what to pick. The reason the square works here is that the nonlinearity in the equation is a square. When people say EDMD is finicky, what they usually mean is that they chose a dictionary by convenience rather than by looking at the equations.

Second, once the model is linear you get the whole linear toolbox back: pole placement, LQR, Kalman filtering, all the rank tests from Part V. That is the practical prize, and it is why Koopman methods keep appearing in fault detection work. A linear observer on a lifted space can do things a linearized observer on the original space cannot, because the lift is exact rather than a first order truncation valid near one operating point.

The honest limit

Exact finite closure is rare. Most systems have no finite invariant subspace, and any practical dictionary is an approximation whose quality varies over the state space. The strong claim, that any nonlinear system becomes a finite linear one, is false. The useful claim, that a well chosen small lift often buys a great deal, is true and testable, and the test is the figure above.

33

Symbolic regression and the Pareto front

SINDy fixes the library and picks coefficients. Symbolic regression drops that restriction and searches over the space of expression trees themselves, which is enormous, discontinuous, and cannot be gradient descended. Modern implementations use evolutionary search, and PySR is the one most people reach for [25]. The same group showed that constraining a network's architecture first and reading expressions out of its learned components recovers known force laws and generalizes better than the network it came from [15].

The output is not one equation. It is a front: for each level of complexity, the most accurate expression found at that complexity. Reading that front is the skill, because the tempting move, taking the most accurate one, is almost always wrong.

Here is a real front, on data you have already met. Figure 10 showed the pendulum period error against amplitude, and the exact answer involves a complete elliptic integral. What is the best short formula for it?

Figure 28

Accuracy against complexity, and the knee

on the Pareto front dominated Taylor series truncations
Eleven candidate forms for the pendulum period ratio, each scored by root mean square error against the exact elliptic integral over amplitudes from 1 to 137 degrees, and by a simple node count for complexity. Hover any point for the expression. Coefficients marked as fitted were obtained by least squares; the rest are the exact series coefficients.

The result is a small lesson in why this method exists. The four term Taylor series, at complexity 13, achieves a root mean square error of 8.2 thousandths. A simple rational form with one fitted coefficient, at complexity 6, achieves 7.4 ten-thousandths. Less than half the complexity, and eleven times more accurate.

Taylor expansion could never have produced that form, because it is not a polynomial. The search found it because the search was not restricted to polynomials. That is the entire argument for symbolic regression in one comparison, and it is also the argument against trusting a series expansion just because it came from an exact derivation.

How to read a Pareto front

Look for the knee, the point after which more complexity buys very little accuracy. Prefer forms whose parts have physical meaning over forms that merely fit. Be suspicious of a large jump in accuracy at high complexity on noisy data, because that is usually the search fitting the noise. And check the found expression's behaviour outside the data range before you use it there, since nothing in the fit constrained it.

34

Physics informed networks, and the through line

The fourth approach runs the other direction. Rather than extracting an equation from data, a physics informed neural network takes an equation you already have and puts it into the training objective. The network is trained to match whatever data exists and to make the residual of the governing differential equation small at points where there is no data at all, with those residual derivatives supplied by automatic differentiation [13].

The symbolic equation never becomes a number here. It stays a statement about what any admissible solution must satisfy, and it acts as an infinite supply of free training data in the regions your sensors do not reach. That is its real value: extrapolation constrained by physics rather than by the shape of the data.

MethodWhat you supplyWhat comes outFails when
SINDyData, plus a library of candidate termsA short, readable equationThe true term is not in the library, or the derivatives are too noisy
EDMD / KoopmanData, plus a dictionary of observablesA linear operator on the lifted spaceNo finite dictionary closes, so the lift only approximates
Symbolic regressionData, plus a set of allowed operatorsA front of expressions, accuracy against complexityThe search space is too large, or the front's high end is fitting noise
PINNThe governing equation, plus sparse dataA network that satisfies bothThe equation is stiff or multiscale, where the loss is hard to balance

The through line is the same in all four. A purely learned model that cannot be inspected is hard to certify and harder to argue with. A purely first principles model that ignores fleet data is usually wrong at the edges, which is exactly where the failures live. Each of these methods is an attempt to keep a readable symbolic object while letting data supply what derivation cannot.

Part X

Root cause and prognosis

Everything above exists to answer two questions in the field: what broke, and how long have I got.

35

Where each side belongs

A diagnostic or prognostic system is a chain, and the two sides of this page do not compete across it. They occupy different links, and putting either one in the wrong link is the usual reason these systems disappoint.

The short version: symbolic work decides what to compute, numeric work computes it. Every question of the form "can this be detected", "which faults can be told apart", "which sensor is worth adding", "what does this parameter do to remaining life" is a question about structure, and structure lives in the letters. Every question of the form "is it happening now, on this vehicle, at this mileage" is a question about numbers.

Figure 29

The diagnosis and prognosis chain, split by which side owns it

The left column is done once, offline, in symbols, and it decides what the right column will be able to see. Almost every field diagnostic that cannot isolate a fault, or that reports a remaining life nobody believes, has an unexamined left column.
In plain words

You cannot detect what the model structure hides, and no amount of data will rescue you. That is not a pessimistic claim, it is a checkable one, and Sections 18 and 20 already showed how to check it before any hardware exists.

36

Residuals, derived rather than tuned

A residual is a quantity that is zero when the system is healthy and non-zero when it is not. The naive way to build one is to run a model alongside the real thing and subtract. That works, and it requires knowing every state, including the ones you do not measure.

The better way is to eliminate the unmeasured states symbolically, leaving a relation among measured signals only. In the fault diagnosis literature this is an analytical redundancy relation [31], and building one is exactly the polynomial elimination from Section 6. Isermann's survey is the standard map of the model-based family this belongs to [35].

Take the two node thermal model. Solve the second equation for the unmeasured core temperature and substitute into the first:

SymPyT1_expr = solve(eq2, T1)[0]
R = expand(eq1.lhs.subs(T1, T1_expr) - eq1.rhs.subs(T1, T1_expr))
the resulting relation, cleared of denominatorsR = C_1*C_2*T2'' + (C_1*k_12 + C_1*k_2a + C_2*k_12)*T2'
    + k_12*k_2a*(T2 - T_a) - k_12*q

Look carefully at what came out. Those coefficients are the denominator of the transfer function from Section 19, applied to the measured temperature, minus the numerator applied to the known input. That is not a coincidence and it is worth stating as a rule: the analytical redundancy relation is the transfer function written as a differential operator and moved to one side. If you already derived one, you already have the other.

Run the healthy model through it and the residual is zero to numerical noise. Run four faults through it, using the healthy parameters in the relation, and every one of them lights it up.

Figure 30

One residual, four faults, and the limit of one sensor

residual, faulted residual, healthy detection threshold
The residual is computed from the case temperature and the known heat input only; the core temperature never enters. All four faults are detected and none is identified, because a single scalar cannot distinguish four causes. Note carefully that this runs on noise-free simulated data: the healthy trace is flat at 1×10⁻⁴ W because there is no sensor noise in it. Put a real sensor on this relation and it stops working entirely, for reasons that turn out to be worth a page of their own. See the note below.
An important correction

The figure above uses noise-free data, and that hides a serious problem. The relation contains a second derivative of the measured temperature, so sensor noise arrives amplified by the square of the sample rate. With a quarter of a degree of noise at two second sampling the residual's noise standard deviation is about 122,000 W against a fault signal of 317 W. As a detector this does not work, and no threshold fixes it.

The information is not lost, only discarded: because the fault and the noise pass through the same operator, the best achievable detection performance is unchanged, and a matched filter or a parameter estimator recovers all of it. That is why the extended Kalman filter of Section 41 succeeds on the same noise where this residual fails. The companion monograph Above the Noise Floor works this through, along with thresholds, build tolerance, sensor faults and the maintenance decision.

Detection is the easy half. The residual said something is wrong and could not say what. For that you need more than one relation, which means more than one sensor, and the question of which sensor is again symbolic.

37

Isolation, and what a second sensor actually buys

Add a core temperature sensor and the model splits into two relations, one per node, each testable on its own:

two analytical redundancy relationsR_a = C_1*T1' + k_12*(T1 - T2) - q
R_b = C_2*T2' - k_12*(T1 - T2) + k_2a*(T2 - T_a)
structural sensitivity: is dR/dtheta identically zero?        R_a  R_b
C_1    1   0
C_2    0   1
k_12   1   1
k_2a   0   1
q      1   0

That table is the fault signature matrix, and it was produced by asking SymPy whether each partial derivative is identically zero. Read the columns and it tells you exactly what the second sensor bought and what it did not.

The way out of the first collision is not another sensor. It is a different test on the sensors you have. At steady state both derivatives vanish, so the first relation collapses to a heat balance in which a capacitance cannot appear at all. Simulating that out to equilibrium confirms it exactly.

Figure 31

Signature matrix, with the steady state column that breaks the tie

Every number is the mean residual from a full simulation of that fault, evaluated with the healthy parameters. The first two columns are the transient signature; the third is the value of the first relation once everything has settled, where a capacitance has nowhere to act. Adding a test costs nothing but patience, and it isolates a pair that a third sensor would also have isolated at considerably greater expense.
The general procedure

Write every relation the model supports. Take the symbolic derivative of each with respect to each fault parameter. Any two faults with identical columns are not isolable, and the fix is either a new relation, a new operating condition, or an admission in the diagnostic report that the two cannot be separated. Deciding this on paper is the whole subject of structural analysis for diagnosis [32], and the search for minimal testable subsystems in a large model has been mechanized [36]. It is the cheapest engineering hour you will spend on a monitor.

38

Four machines, worked

The procedure above is general. Here it is applied to four real components, with the symbolic step written out and the consequence measured.

Figure 32

Four components, four symbolic sensitivities

Each case follows the same path: write the physics, differentiate with respect to the damage parameter, and read where the sensitivity is large, small, or exactly zero. The zeros are the important part. They are the operating conditions under which your monitor is blind, and they are invisible in a numerical model.

The one worth dwelling on

The permanent magnet case produces a result in one line that changes how the monitor is designed. Differentiate the torque equation with respect to magnet flux and you get three halves times pole pairs times the q-axis current, and nothing else. At zero q-axis current that sensitivity is exactly zero, so a demagnetization monitor built on torque is blind at cruise, at coast, and at standstill. That is most of a drive cycle.

The back-EMF route has sensitivity equal to electrical speed instead, which is non-zero whenever the machine turns. The two routes cover complementary parts of the operating envelope, and knowing that requires exactly one symbolic differentiation of a formula you already had.

0
torque sensitivity to magnet flux at iq = 0
24.36° → 27.14°
MTPA angle drift at 80 percent magnet health
15.7%
peak torque lost at 80 percent health
0.21%
extra loss from not retuning the angle

That last pair of numbers is a warning about acting on the wrong signal. The optimal current angle moves measurably as the magnet weakens, which makes it a decent health feature. But failing to retune it costs only two tenths of a percent of torque, because the torque curve is flat near its peak. The angle drift is worth measuring and not worth correcting, and only the symbolic form makes that distinction visible.

39

When the controller hides the fault

Everything so far assumed you observe the plant. On a modern vehicle you almost never do. You observe a plant wrapped in a controller that has been designed, with considerable care, to make faults invisible.

Consider an actuator whose gain degrades slowly, inside a loop with integral action. Work out the closed loop symbolically and the problem states itself:

closed loop, outputT(s) = g*(K_i + K_p*s) / (s**2 + a*s + K_p*g*s + K_i*g)
DC gain T(0) = 1
closed loop, control effortU(s)/R(s) = (K_i*a + (K_i + K_p*a)*s + K_p*s**2) / (same denominator)
DC gain U(0) = a / g

The output gain is exactly one for any positive actuator gain. That is what integral action is for. It also means the tracking error, the residual everybody reaches for first, contains no information about the fault whatsoever. Meanwhile the control effort goes as one over the gain, and has been reporting the degradation faithfully since the first hour.

Figure 33

Two thousand three hundred hours of silence

control effort, steady state tracking error actuator gain, the truth
A simulated actuator losing gain linearly over 4,000 hours inside a PI loop with the command limited to one. Until the command saturates at about 2,300 hours the tracking error stays below 7×10⁻¹¹, which is numerically zero, while the control effort climbs steadily from 0.50 to 0.99. After saturation the fault appears abruptly and looks like a step change rather than the two thousand hours of wear it actually is.
What this means for a fleet monitor

Any diagnostic built on tracking error inside a loop with integral action will report nothing until the actuator runs out of authority, and will then report a sudden failure. The onset date in the warranty record will be wrong by the entire degradation period. Log the control effort, the duty cycle, the adaptation and trim values, and the saturation flags. Those are the signals the loop cannot suppress, and the symbolic transfer function tells you which they are before you have to guess.

40

Prognosis, route one: the damage law inverted

Prognostics splits into two approaches that get conflated constantly. This section is the first: you know the physics of the degradation, you write the rate law, and you integrate it to a threshold. This is the physics-of-failure route, and where the integral has a closed form, the answer is a formula rather than a simulation.

Crack growth is the standard example. Paris and Erdogan's 1963 law says the crack extension per cycle is a power of the stress intensity range [33]. Substituting the stress intensity and integrating is one line:

SymPydK = Y*Delta_sigma*sqrt(pi*a)
N_f = integrate(1/(C*dK**m), (a, a_0, a_f))
Output, m != 2 branchN_f = 2*(a_0**(1 - m/2) - a_f**(1 - m/2))
      / ( C*(m - 2)*(sqrt(pi)*Delta_sigma*Y)**m )

SymPy returns a piecewise result and flags the m = 2 case separately, where the integral becomes a logarithm. That is the tool being careful on your behalf about a special case most textbook derivations skip.

With a representative steel the closed form gives 780,125 cycles, which agrees with numerical quadrature of the same integral to sixteen digits. The number is not the interesting part. This is:

Figure 34

What the exponent tells you before you run anything

Life against detection threshold and against tolerated final crack length, from the closed form. For an exponent above two the integral is dominated by the small crack end, so what you can detect matters enormously and what you will tolerate matters hardly at all. Halving the detectable crack from 0.5 to 0.2 mm adds 73 percent to useful life; doubling the tolerated final crack from 12 to 25 mm adds 8 percent.

The same structure appears everywhere in reliability, which is why it is worth recognizing. Every one of the standard damage laws is a power law or an exponential, from Archard's wear relation [37] to the Coffin-Manson low cycle fatigue law [38], and the log derivative is therefore a constant that you can read directly off the symbol without simulating anything.

Component and mechanismLawd ln(life) / d ln(stress)Life at +10% stress
Gear tooth bending fatigueN ~ S^-b, b ≈ 9-9.000.424×
Rolling bearing spallL₁₀ = (C/P)^p, p = 3-3.000.751×
Crack growth, ParisN ~ Δσ^-m, m = 3-3.000.751×
Solder joint, Coffin-MansonN ~ ΔT^-c, c ≈ 2-2.000.826×

Read that table across and it says a ten percent load reduction on a gear buys two and a half times the life, while the same reduction on a bearing buys about 1.4 times. Nobody simulated anything. The exponent is the engineering answer, and it exists only because the law was kept symbolic.

Temperature, and the trap in averaging it

For thermally activated mechanisms the knob is temperature and the law is exponential, which makes averaging dangerous in a specific way. Take a duty cycle with 55 percent of the time at 45 degrees, 30 at 70, 12 at 95 and 3 at 120. The arithmetic mean is 60.75 degrees. The damage-equivalent constant temperature, the one that produces the same integrated Arrhenius damage, is 76.89 degrees.

Using the arithmetic mean understates the damage rate by a factor of three. The three percent of time spent at 120 degrees does more harm than the fifty five percent at 45, and the only reason anyone knows that is that the rate law was integrated rather than averaged.

The running thermal example inverts just as cleanly. Let the heat sink foul exponentially, take the steady state core temperature, and solve for the time at which it reaches a limit:

SymPyk_2a(t) = k_0*exp(-t/tau)
T1_ss = T_a + q/k_2a(t) + q/k_12
solve(Eq(T1_ss, T_limit), t)
RUL, closed formt_f = tau*log( k_0*(k_12*(T_limit - T_a) - q) / (k_12*q) )

# 2,722.6 hours for the nominal values, matching
# a numerical scan of the same relation to 0.007 h
41

Prognosis, route two: estimate the damage as a state

The second route applies when you do not know the degradation trajectory, only that some parameter is drifting. You promote that parameter to a state, let it random walk, and estimate it jointly with the physical states from whatever you can measure. Then you project the estimate forward to the threshold.

The question to answer first is whether the augmented system is observable at all, and that is symbolic. Take Lie derivatives of the measurement along the augmented dynamics and check the rank:

SymPy, states [T1, T2, k_2a], measuring T2 onlyL = [h, L_f h, L_f^2 h]
O = Matrix([[diff(l, s) for s in states] for l in L])
O.rank()
Outputrank = 3 of 3 -> observable

det(O) = k_12*(C_1*k_12*(T1-T2) - C_1*k_2a*(T2-T_a)
        + C_2*k_12*(T2-T_a)) / (C_1*C_2**3)

This is worth reconciling with Section 20, which said the parameters were not identifiable from the case temperature. Both statements are true and the difference is what is held known. Section 20 left all four parameters free and had three transfer function coefficients to determine them. Here the two capacitances are known from mass and specific heat, and only the conductance drifts, so there is information to spare. Fixing a parameter from an independent source, which Section 20 recommended, is precisely what makes the rest recoverable, and the rank test proves it rather than assuming it.

With observability settled, the filter is ordinary. The Jacobian it needs is the symbolic one, extended by a column for the parameter.

Figure 35

An extended Kalman filter tracking a parameter it cannot see

truth filter estimate ±3σ band
2,600 hours of hourly measurements of case temperature only, with 0.25 K of noise. The conductance to ambient falls from 2.500 to 2.195 W/K and is never measured. After settling, the filter tracks it with a root mean square error of 0.0022 W/K, and its estimate of the unmeasured core temperature ends within 0.08 K of the truth. Switch to the core temperature view to see the state the whole exercise exists to protect.
Choosing between the two routes

Route one when you know the mechanism and can write its rate law: it extrapolates honestly beyond the data, because the physics constrains the shape. Route two when you know the model but not the degradation trajectory: it tracks whatever is actually happening, but projecting it forward assumes the drift continues as observed, which is an assumption and not a physical law. In practice they compose: estimate the damage state with route two, then extrapolate it with the rate law from route one instead of with a straight line.

42

The number, and the confidence you can put on it

A remaining useful life with no interval on it is not an engineering result. The symbolic form gives you two ways to attach one, and they are complementary rather than competing [34].

The first is the delta method: differentiate the closed form with respect to every uncertain input, square, weight by each variance, and add. This is one symbolic Jacobian and it is essentially free. Its real product is not the number at the bottom, it is the ranking.

Figure 36

Where the uncertainty in the life estimate actually comes from

Variance contributions from the closed form RUL of Section 40, and the Monte Carlo distribution of the same expression over 200,000 samples. The heat load alone accounts for half the variance in the answer. Tightening the estimate of the fouling time constant, which is the parameter most people would instinctively chase, would barely move the interval at all.

The ranking is the actionable output. Half the variance in this life estimate comes from the heat load, and a further eighteen percent from the temperature limit. The degradation time constant, the parameter that a prognostics project would naturally spend its effort on, contributes three percent. That reordering of priorities came from one differentiation of a formula, and it would have taken a large sampling campaign to discover numerically.

The second method is Monte Carlo through the same closed form, which is cheap precisely because the expression is closed form. Here it gives a standard deviation of 1,550 hours against the delta method's 1,530, so the linearization was accurate to about one percent.

Do not over-read that agreement

The delta method assumes the function is nearly linear across the uncertainty range. Here it is, so the two agree. When the uncertainty is large enough to reach a region where the expression curves sharply, or near a point where a denominator approaches zero, the delta method quietly fails and Monte Carlo does not. The honest practice is to use the Jacobian to rank the drivers, sample for the interval you will actually act on, and compare the two as a check. Both are cheap once the expression is symbolic; neither is available if the model only exists as a solver.

And a last word on the interval itself. The nominal answer here is 2,723 hours, and the fifth to ninety-fifth percentile range is roughly 140 to 5,230 hours. That is a very wide interval on a very confident-looking number, and it is the correct answer given the stated input uncertainties. A prognostic that reports 2,723 hours without it is not more accurate. It is less honest.

Part XI

Practice

Where it came from, and what to actually do on Monday.

43

A short history

Symbolic computation is older than most of the numerical tools it now feeds. It was, in fact, one of the first serious applications of artificial intelligence research.

Figure 37

Sixty five years of symbolic and numeric tools

Hover any marker for detail. The symbolic line starts first, in the artificial intelligence labs of the early 1960s. The numeric Python line starts much later and grows much faster, and the two only really meet once SymPy and NumPy end up importable in the same file.

The first milestone is worth a sentence of its own. In 1961 James Slagle's SAINT program at MIT solved freshman calculus integration problems by heuristic search, and it was presented as artificial intelligence, because that is what it was. Macsyma grew out of that line of work at MIT's Project MAC from 1968 onward, and Joel Moses, who wrote its integration engine, later published a personal history of the project [16]. Nearly every CAS alive today is downstream of Macsyma, either technically or intellectually.

44

A recipe you can follow

For a physics based model that has to end up as running code, this ordering works and the reverse does not.

  1. Write the physics as symbols, with assumptions. Declare masses positive, temperatures real, angles real. The assumptions are not decoration; they are what lets simplification proceed.
  2. Non-dimensionalize before anything else. Section 16. Find out how many parameters you really have before you start sweeping seven of them.
  3. Derive. Apply Euler-Lagrange, take the energy balance, apply the coordinate transform. Let the tool do the algebra you would otherwise do on paper.
  4. Run the five checks. Degenerate case, equilibrium sign, conservation, dimensions, limits. Section 13. Nothing proceeds until these pass, because everything downstream inherits the error.
  5. Ask what the model can and cannot tell you. Controllability, observability and structural identifiability are symbolic questions with symbolic answers, and answering them now is far cheaper than discovering the answer from a calibration that will not converge. Sections 18 and 20.
  6. Simplify with intent. Use the targeted function, not the general one. Stop when the expression is small enough to read.
  7. Take the derivatives you will need later. Jacobians for the estimator, gradients for the optimizer, sensitivities for identifiability. Do it now, while everything is still symbolic.
  8. Choose the numerically stable form, not the shortest one. Section 17. Then discretize, remembering that the exact matrix exponential wants numbers, not symbols. Section 21.
  9. Reduce and generate. cse, horner, then lambdify for Python or codegen for C. Check the operation count fell.
  10. Cross the border once. From here on, arrays and floats. Do not go back into the symbolic layer inside a loop.
  11. Keep the derivation script in version control next to the generated code. The generated file is an artifact, not source. When a parameter definition changes, you regenerate, you do not patch.
  12. If it will be monitored, do the diagnosis work while you are still in symbols. Eliminate the unmeasured states into a redundancy relation, take the signature matrix, and find out which faults are indistinguishable before anyone specifies a sensor. Sections 36 and 37.
  13. If it will be forecast, invert the damage law before you simulate it. A closed form time-to-threshold gives you the sensitivity ranking for free, which is what tells you where to spend measurement effort. Sections 40 and 42.
The one line summary

Derive symbolically, prove it on a degenerate case, check what the model can identify, generate code, and never call subs in a loop.

45

Traps that bite everyone

SymptomCauseFix
Symbolic code is unbearably slow on real dataEvaluating with subs inside a looplambdify once, then call the compiled function on arrays
An obviously true simplification will not happenMissing assumptions, so the identity is not actually always trueDeclare positive=True, real=True, integer=True as appropriate
Exact cancellation stops working partway throughA literal float such as 0.5 leaked into the expressionUse sp.Rational(1,2) and keep floats out until the very end
Expression grows until the session hangsExpression swell, usually from repeated differentiation or a matrix exponentialSimplify or cse at each stage, or substitute numbers before the step that swells
Generated function rejects an arraylambdify defaulted to the standard library math modulePass 'numpy' explicitly as the modules argument
An integral comes back unevaluatedQuite possibly no elementary antiderivative existsConfirm with a different system, then switch to numerical quadrature
Solving returns CRootOf objectsDegree five or higher; no formula in radicals existsThat is the exact answer. Evaluate it numerically to whatever precision you need
Two systems disagree on a simplified formBoth are right; they picked different canonical formsSubtract and simplify the difference, or evaluate both at random points and compare
The Jacobian looks right but the filter divergesDerivative taken with respect to the wrong variable orderingBuild the state vector explicitly as a Matrix and pass it to jacobian
The fit converges beautifully but the parameters are nonsenseStructural non-identifiability. Several parameter sets give the same outputRun the identifiability check first. Add a sensor, fix a parameter independently, or reparameterize
A discrete model goes unstable when the plant obviously does notForward Euler at too long a sample periodUse the exact discretization or Tustin, both unconditionally stable here
A component model will not initialize, or drifts off its constraintHigh index DAE needing consistent initial conditionsIndex reduction, which the tool can usually do once you tell it to
A monitor detects a fault but always reports the wrong componentTwo faults share a column in the signature matrixCompare columns symbolically. Add a relation, a sensor, or a steady state test, or report the ambiguity honestly
A degradation appears suddenly after years of nothingClosed-loop masking. The controller absorbed it until it ran out of authorityMonitor control effort, adaptation values and saturation flags, not tracking error
A health monitor works on the dyno and never triggers in the fieldThe sensitivity is zero over the operating conditions the vehicle actually seesDifferentiate the observable with respect to the damage parameter and check where it vanishes
The remaining useful life estimate is confident and wrongNo interval was propagated, or one dominant input was never characterizedSymbolic Jacobian for the variance ranking, Monte Carlo for the interval you act on
Generated C disagrees with the Python it came fromPrecision, or an integer division, or a different pow conventionCross check both against random inputs across the envelope, in the target's precision
46

Glossary

Symbolic computation
Manipulating mathematical expressions as structured objects, exactly, with the letters intact.
CAS
Computer algebra system. Any program whose primary data type is an expression rather than a number.
Expression tree
The internal representation of an expression: operations as nodes, operands as children.
Canonical form
A standard shape every equivalent expression is rewritten into, so equality can be tested by comparison.
Expression swell
The tendency of symbolic results to grow rapidly in size during a derivation, even when the final answer is small.
Catastrophic cancellation
Loss of all significant digits when two nearly equal floating point numbers are subtracted.
Condition number
How much a small relative change in the inputs can be amplified in the output. A property of the formula you chose, not only of the problem.
Grobner basis
A rewritten form of a system of polynomial equations that makes solving and variable elimination mechanical.
Risch algorithm
A decision procedure for whether an elementary function has an elementary antiderivative, and for finding it if so.
Buckingham Pi theorem
A relationship among n variables built from r independent dimensions can be written with exactly n minus r dimensionless groups.
Structural identifiability
Whether parameters can be recovered uniquely from perfect noise free data with the sensors you have. A property of the model, answerable before any measurement.
Differentiation index
How many times the constraints of a differential algebraic system must be differentiated before it becomes solvable as an ordinary differential equation.
Acausal modeling
Declaring physical relationships without fixing what is input and what is output, and letting the tool derive a solvable form.
Jacobian
The matrix of first partial derivatives of a vector valued function. The central object in estimation, optimization and linearization.
Lambdify
SymPy's function that compiles a symbolic expression into a plain numeric function. The exit door from the symbolic world.
Common subexpression elimination (cse)
Factoring repeated subtrees out into temporary variables before generating code.
Horner form
A polynomial written as nested multiplication, so it evaluates with only multiplies and adds and no exponentiation.
Vectorization
Applying an operation to a whole array in compiled code instead of looping in the interpreter.
Broadcasting
NumPy's rule for combining arrays of different but compatible shapes without copying data.
Automatic differentiation
Computing exact derivative values by propagating derivative information through a numeric program, without forming a symbolic formula.
Koopman operator
The linear operator describing how functions of the state, rather than the state itself, evolve. Infinite dimensional in general, but sometimes closing on a small finite set.
Observable (Koopman sense)
Any scalar function of the state. The dictionary of observables is what an EDMD model is built on, and choosing it is a modeling decision.
Analytical redundancy relation
A relation among measured signals only, obtained by eliminating the unmeasured states symbolically. Zero when healthy, non-zero when not.
Residual
The evaluated value of such a relation on real data. Detection is a test on its magnitude; isolation is a test on the pattern across several residuals.
Fault signature matrix
Which residuals respond to which faults. Two faults with identical columns cannot be told apart, whatever algorithm is applied afterward.
Damage-equivalent temperature
The single constant temperature producing the same integrated Arrhenius damage as a varying duty cycle. Always higher than the arithmetic mean.
Remaining useful life (RUL)
Time or cycles from now until a damage state reaches its threshold. An RUL quoted without an interval is not an engineering result.
Delta method
First-order uncertainty propagation: square the symbolic derivatives, weight by input variances, and add. Cheap, and it ranks the drivers.
Pareto front
The set of candidate models for which nothing simpler is also more accurate. The output of a symbolic regression search.
47

References

  1. Goldberg, D. (1991). What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys, 23(1), 5-48. dl.acm.org/doi/10.1145/103162.103163
  2. Buchberger, B. (1965/2006). An algorithm for finding the basis elements of the residue class ring of a zero dimensional polynomial ideal. Doctoral thesis, University of Innsbruck; English translation in Journal of Symbolic Computation, 41(3-4), 475-511. sciencedirect.com/science/article/pii/S0747717105001483
  3. Risch, R. H. (1969). The problem of integration in finite terms. Transactions of the American Mathematical Society, 139, 167-189. See also Risch (1970), The solution of the problem of integration in finite terms, Bulletin of the AMS, 76(3), 605-608. projecteuclid.org/euclid.bams/1183531821
  4. Park, R. H. (1929). Two-reaction theory of synchronous machines: generalized method of analysis, part I. Transactions of the AIEE, 48(3), 716-727. doi:10.1109/T-AIEE.1929.5055275
  5. Baydin, A. G., Pearlmutter, B. A., Radul, A. A., and Siskind, J. M. (2018). Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research, 18(153), 1-43. jmlr.org/papers/v18/17-468.html
  6. Wolfram, S. (2026). Launching Version 15 of Wolfram Language and Mathematica. Published 16 June 2026. writings.stephenwolfram.com
  7. Meurer, A., et al. (2017). SymPy: symbolic computing in Python. PeerJ Computer Science, 3, e103. peerj.com/articles/cs-103
  8. Harris, C. R., et al. (2020). Array programming with NumPy. Nature, 585, 357-362. nature.com/articles/s41586-020-2649-2
  9. Virtanen, P., et al. (2020). SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature Methods, 17, 261-272. nature.com/articles/s41592-019-0686-2
  10. Andersson, J. A. E., Gillis, J., Horn, G., Rawlings, J. B., and Diehl, M. (2019). CasADi: a software framework for nonlinear optimization and optimal control. Mathematical Programming Computation, 11, 1-36. link.springer.com/article/10.1007/s12532-018-0139-4
  11. Modelica Association. Modelica Language Specification. specification.modelica.org
  12. Bradbury, J., et al. (2018). JAX: composable transformations of Python and NumPy programs. github.com/jax-ml/jax
  13. Raissi, M., Perdikaris, P., and Karniadakis, G. E. (2019). Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686-707. sciencedirect.com/science/article/abs/pii/S0021999118307125
  14. Brunton, S. L., Proctor, J. L., and Kutz, J. N. (2016). Discovering governing equations from data by sparse identification of nonlinear dynamical systems. PNAS, 113(15), 3932-3937. pnas.org/doi/10.1073/pnas.1517384113
  15. Cranmer, M., et al. (2020). Discovering symbolic models from deep learning with inductive biases. Advances in Neural Information Processing Systems 33. arxiv.org/abs/2006.11287
  16. Moses, J. (2012). Macsyma: a personal history. Journal of Symbolic Computation, 47(2), 123-130. sciencedirect.com/science/article/pii/S0747717110001483
  17. Slagle, J. R. (1961). A heuristic program that solves symbolic integration problems in freshman calculus. Doctoral thesis, Massachusetts Institute of Technology.
  18. SymPy documentation. Simplification, assumptions, code generation, and the physics.units module. docs.sympy.org
  19. NumPy documentation. Broadcasting. numpy.org/doc/stable/user/basics.broadcasting.html
  20. Wolfram Research. Wolfram Language and System Documentation Center. reference.wolfram.com/language
  21. IEEE Standard for Floating-Point Arithmetic, IEEE 754. The double precision format underlying every numeric result on this page, with a 53 bit significand giving roughly 15.95 decimal digits.
  22. Buckingham, E. (1914). On physically similar systems: illustrations of the use of dimensional equations. Physical Review, 4(4), 345-376. link.aps.org/doi/10.1103/PhysRev.4.345
  23. Ljung, L. and Glad, T. (1994). On global identifiability for arbitrary model parametrizations. Automatica, 30(2), 265-276. sciencedirect.com/science/article/abs/pii/0005109894900299
  24. Pantelides, C. C. (1988). The consistent initialization of differential-algebraic systems. SIAM Journal on Scientific and Statistical Computing, 9(2), 213-231. epubs.siam.org/doi/10.1137/0909014
  25. Cranmer, M. (2023). Interpretable machine learning for science with PySR and SymbolicRegression.jl. arxiv.org/abs/2305.01582
  26. Koopman, B. O. (1931). Hamiltonian systems and transformation in Hilbert space. PNAS, 17(5), 315-318. pnas.org/doi/10.1073/pnas.17.5.315
  27. Williams, M. O., Kevrekidis, I. G., and Rowley, C. W. (2015). A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. Journal of Nonlinear Science, 25, 1307-1346. link.springer.com/article/10.1007/s00332-015-9258-5
  28. Mezic, I. (2005). Spectral properties of dynamical systems, model reduction and decompositions. Nonlinear Dynamics, 41, 309-325.
  29. Higham, N. J. (2002). Accuracy and Stability of Numerical Algorithms, 2nd edition. SIAM. The standard reference for conditioning and for why algebraically equal formulas are not numerically equal.
  30. Mattsson, S. E. and Soderlind, G. (1993). Index reduction in differential-algebraic equations using dummy derivatives. SIAM Journal on Scientific Computing, 14(3), 677-692.
  31. Gertler, J. (1998). Fault Detection and Diagnosis in Engineering Systems. Marcel Dekker. The standard treatment of analytical redundancy relations and structured residual design.
  32. Blanke, M., Kinnaert, M., Lunze, J., and Staroswiecki, M. (2016). Diagnosis and Fault-Tolerant Control, 3rd edition. Springer. Structural analysis, isolability, and sensor placement worked as a systematic method.
  33. Paris, P. C. and Erdogan, F. (1963). A critical analysis of crack propagation laws. Journal of Basic Engineering, 85(4), 528-534. doi:10.1115/1.3656900
  34. Sankararaman, S. and Goebel, K. (2015). Uncertainty in prognostics and systems health management. International Journal of Prognostics and Health Management, 6(4). papers.phmsociety.org/index.php/ijphm/article/view/2319
  35. Isermann, R. (2005). Model-based fault-detection and diagnosis: status and applications. Annual Reviews in Control, 29(1), 71-85. sciencedirect.com/science/article/abs/pii/S1367578805000052
  36. Krysander, M., Aslund, J., and Nyberg, M. (2008). An efficient algorithm for finding minimal overconstrained subsystems for model-based diagnosis. IEEE Transactions on Systems, Man and Cybernetics, Part A, 38(1), 197-206. The computational core of structural fault diagnosis.
  37. Archard, J. F. (1953). Contact and rubbing of flat surfaces. Journal of Applied Physics, 24(8), 981-988. The wear law used in the thrust bearing case of Section 38.
  38. Coffin, L. F. (1954) and Manson, S. S. (1953), independently. The low cycle fatigue relation between plastic strain range and cycles to failure, used here in its temperature-swing form for solder joints.
On the numbers in this page

Every symbolic result shown was produced by running the stated input through SymPy 1.14.0 and copying the output verbatim. Every timing, error, identifiability and recovery figure was measured with NumPy 2.4.4 and SciPy on the machine that generated this page and will differ on yours; the orders of magnitude are the point, not the digits. The SINDy, EDMD and Pareto results in Part IX come from complete runs of those methods, not from illustrative values, and so do the residuals, signature matrices, closed-form lives, filter traces and uncertainty budgets in Part X.