Interactive Monograph · Generative Machine Learning
A neural network that learns to turn noise into data by mastering one humble skill: removing a little noise at a time. Start from static, denoise step by step, and a sample of something realistic appears. This monograph builds the idea from scratch, with a model that trains live in your browser, then puts it to work generating and guarding automotive signals.
Imagine slowly stirring a drop of ink into water until the picture it once formed is gone, just uniform gray. That destruction is easy and needs no intelligence. The hard and useful direction is the reverse: starting from gray and running the stirring backward until a picture reassembles. A diffusion model learns exactly that reverse step.
Training has two halves. In the forward process we take real data and corrupt it with Gaussian noise in many small steps, all the way to pure static. This is fixed and hand-defined, no learning involved. In the reverse process a neural network is trained to undo one small step of that corruption: given a noisy example, predict the noise that was added. Chain those learned removals together and you can walk from random static back to a brand-new, realistic sample.
A diffusion model is trained to remove a little noise at a time. Generation is just doing that repeatedly, starting from pure noise, until a fresh sample emerges.
Why bother with tiny steps instead of one big leap? Because each small denoising is an easy problem the network can learn well, and hundreds of easy steps compose into a very hard one: conjuring structured data from nothing. This is what powers modern image and audio generators, and, as the later sections show, what can synthesize rare sensor signals and flag abnormal ones in a vehicle.
The forward process is a fixed recipe. At each step it shrinks the data slightly and adds a pinch of Gaussian noise. A schedule of numbers, written āḷ (a-bar at step t), tracks how much of the original signal survives: it starts near 1 (all signal) and slides to 0 (all noise). A single formula jumps straight to any step:
Drag the slider to run it on a real two-dimensional dataset, a spiral. As the step rises the spiral loses its shape and melts into a featureless blob of noise. That end state, pure Gaussian static, is exactly where generation will later begin.
Here is the whole training loop. Take a clean example, jump it to a random noise level with the formula above, and hand the noisy result to the network along with which step it came from. The network's job is to predict the noise that was added. Compare its guess to the true noise, and nudge its weights to do better. That is the entire objective.
This is the real thing, a small denoiser built from scratch with no libraries. It learns to turn noise into points that lie on the spiral. Press train and watch the loss fall and the sample cloud snap onto the target shape. Then press sample to animate a fresh cloud of noise being denoised, step by step, into the spiral.
Everything you just watched scales up. Replace the two-dimensional spiral with the pixels of an image or the samples of a sensor waveform, swap the small network for a larger one, and the identical loop produces photographs, audio, or the automotive signals in the sections ahead.
The network has one job: read a noisy example and the step number, output the noise to remove. Two design choices matter. First, it must know how noisy the input is, so the step t is turned into a smooth time embedding (a set of sine and cosine features) and fed in alongside the data. Second, its shape follows the data: a U-Net for images, a one-dimensional convolutional or transformer network for signals, and a plain multilayer network for low-dimensional data like our spiral.
There is an equivalent and illuminating way to read the network. Predicting the noise is the same as pointing toward higher-probability data, the score, the direction in which the data becomes more likely. Sampling then looks like repeatedly nudging a point uphill toward the data distribution while shaking it a little. This score-based picture is what connects diffusion to a large body of older statistical theory, and it is the basis of the anomaly detector in the inverter section.
Three practical variants matter in the field:
Because predicting the noise is the same as pointing toward the data, a trained model defines a field of arrows over the whole space, each showing the direction it would push a point to make it more realistic. Here is that field for a model trained on the spiral. Slide the noise level: at low noise the arrows hug the spiral tightly; at high noise they sweep broadly inward from everywhere. Sampling is just following these arrows from a random start.
How many denoising steps do you actually need? Fewer steps generate faster but can lose fidelity. Compare the two samplers on the spiral: DDIM is deterministic and stays clean even at few steps, while DDPM injects fresh randomness at every step, visible as extra scatter. The number is the mean distance of the samples to the true spiral, lower is tighter.
Discriminative models answer questions about data you already have. A generative model like a diffusion model produces new data, and for a drivetrain program that unlocks a different set of jobs, especially in health management where the interesting events are rare.
A battery's most valuable data is its rarest: cells driven to the edge, unusual fault signatures, deep aging. A diffusion model trained on measured discharge curves learns the shape of normal behavior and can generate new, realistic voltage-under-load traces to enrich a diagnostic dataset. The same forward process from section two applies directly to a real battery signal, watch a discharge trace dissolve into noise.
Turned around and conditioned on the recent past, a diffusion model becomes a probabilistic forecaster: instead of a single predicted state-of-charge or remaining-life number, it samples many plausible futures, and their spread is a calibrated uncertainty band, which is what a battery management system actually needs for a safe margin.
This is real. A compact diffusion model was trained on discharge curves, and here it forecasts the rest of a curve from its first half. The solid black line is what has been observed so far; press forecast and the model draws many continuations past the marker, all consistent with the history but spreading apart into the future. That spread is the uncertainty, and the true continuation (dashed teal) should sit inside it.
The score view from section six gives a clean anomaly detector. Train a diffusion model on healthy inverter telemetry only, junction-temperature traces, current spectra, gate signals. Healthy data is easy for the model to denoise because it lies where the model has seen data. A degrading module produces telemetry the model has never learned, and its denoising error stays stubbornly high. That error is the anomaly score, and it needs no examples of the fault to work.
Here the detector is real. A diffusion model trained only on healthy phase current scores a live signal by how badly it can denoise it. Drag the slider to inject a third-harmonic distortion, the signature of a developing fault. The waveform bends, and the anomaly score, which the healthy-trained model cannot help but report, climbs from calm into the alarm zone. No fault example was ever needed to set this up.
Diagnostics for a PMSM lean on the shape of its phase currents: a healthy machine gives clean sinusoids, while faults like inter-turn shorts or demagnetization stamp characteristic harmonics onto them. To train a robust classifier you need many examples of each, and the faulty ones are hard to obtain. A diffusion model trained on current signatures can manufacture more.
The generator below is real, and it is conditioned on a fault class. One compact diffusion model was trained on labeled phase-current signatures, healthy, demagnetization, and inter-turn short, and its learned weights run right here. Pick a class and it samples a fresh waveform of exactly that type from pure noise. Every trace is newly synthesized, not replayed.
That is the whole augmentation loop. The faulty classes, the ones you can rarely collect from real hardware, are generated to order at whatever mix you want, and a classifier trained on the balanced set becomes sensitive to faults it has seldom, or never, physically seen.
The foundational and most useful works, in rough order of the ideas above.
Citations are given by author, year, and title for orientation; consult the originals for exact statements.