← Back to Autonomy

Graph Learning · Merged introduction

Graph Neural Networks

How a neural network learns from data whose most important feature is who is connected to whom — the anatomy, the architectures, and where graph learning fits.

A Plain-Language Monograph

Graph Neural Networks, Explained Simply

Most neural networks expect tidy grids: a row of numbers, a square image, a sequence of words. But much of the world is not a grid. It is a web of things connected to other things. A Graph Neural Network is the tool built for exactly that shape.

By Majid Mazouchi.

01What is a graph, and why bother?

A graph is just two things: a set of nodes (the things) and a set of edges (the connections between them). That is the whole definition. Once you start looking, graphs are everywhere.

A normal neural network struggles here for a simple reason: it needs a fixed, ordered input. But a graph has no natural order and no fixed size. Your friends are not "friend number 1, friend number 2" in any meaningful sequence, and different people have different numbers of friends. A Graph Neural Network, or GNN, is a neural network designed to read this messy, connected structure directly, without forcing it into a grid.

The core promise A GNN learns a useful description (a vector of numbers, called an embedding) for every node, where that description is shaped not just by the node itself, but by the company it keeps.

02The one big idea: talk to your neighbors

If you remember only one sentence from this page, make it this one: a GNN builds an understanding of each node by repeatedly mixing in information from its neighbors.

Think about how you would describe a stranger. You know nothing about them directly, but you learn they spend time with doctors, hang out at hospitals, and read medical journals. You would reasonably guess they work in medicine. You inferred something about the node by looking at its neighborhood. A GNN does this same trick, with math, automatically, for every node at once.

This neighbor-mixing step is called message passing, and it happens in three small moves that repeat:

Do this once, and each node knows about its immediate neighbors. Do it twice, and each node knows about its neighbors' neighbors. Each repeat is one layer, and each layer lets information travel one more step across the graph.

03GNNs vs the models you already know

The fastest way to place a GNN is to line it up against the two network families most people already have a feel for. Each one makes a different assumption about the shape of its input, and that assumption is the whole story.

ModelAssumes data looks likeClassic useWhat "neighbor" means
CNNA grid (fixed rows and columns)ImagesThe pixels in a fixed square around you
RNNA sequence (a fixed left-to-right order)Text, time seriesThe step right before you
GNNA graph (any connections, any size)Networks, molecules, systemsWhoever you happen to be linked to

Seen this way, a CNN and an RNN are really just special cases of the same idea. A grid is a very orderly graph where every node is wired to the neighbors directly around it. A sequence is an even simpler graph, a single chain. The GNN drops the requirement that the connections be regular at all. That is the entire leap: keep the "learn from your neighbors" trick, throw away the assumption that neighbors come in a neat fixed pattern.

04A picture of the data

Here is a tiny social graph. Each circle is a person (a node), each line is a friendship (an edge). The number inside a node is its current feature, a single value standing in for "what we know about this person right now."

A B C D E F
Figure 1. Six nodes, seven edges. Node A (rust) is our "source" of information. Notice C and F are not directly connected to A, so news from A can only reach them after it hops through the middle of the graph. The number of hops needed is exactly why GNNs use more than one layer.

05A worked example you can follow

Let us make the message-passing rule concrete with the simplest possible version. Give every node a single number. We start node A at 0.9 (it carries a strong signal) and every other node at 0.1 (they know almost nothing yet).

Our update rule for one layer is plain arithmetic. Each node's new value is a 50/50 blend of its own value and the average value of its neighbors:

new_value(v) = 0.5 × old_value(v) + 0.5 × average( old_value of v's neighbors )

Take node D, whose neighbors are A and E. Before the first layer, A is 0.9 and E is 0.1, so their average is 0.5. D itself is 0.1. After one layer:

new_value(D) = 0.5 × 0.1 + 0.5 × 0.5 = 0.05 + 0.25 = 0.30

D crept upward, because one of its neighbors (A) carries a strong signal. Run the rule for every node, then repeat. After a few layers the strong signal from A spreads outward and settles into a smooth pattern across the whole graph. That spreading is learning in its rawest form. In a real GNN, those fixed numbers 0.5 are replaced by weights the network learns, and each node holds a whole vector instead of a single value, but the mechanism is exactly what you just did by hand.

Why this is powerful The same learned rule is applied to every node, no matter how many neighbors it has or what order they come in. That is what lets a GNN handle graphs of any shape and size with one compact set of weights.

06From the toy rule to the real equation

The hand-calculation above is a real GNN layer with two things stripped out: the learning, and the bookkeeping for messy graphs. Let us put them back, one piece at a time, until we arrive at the actual Graph Convolutional Network (GCN) layer that people cite in papers. Nothing here is new math, it is the same blend you already did, written compactly.

The toy rule, for one node, was:

h_v(new) = 0.5 · h_v + 0.5 · mean( h_u for each neighbor u )

Three upgrades turn this into the real thing:

Stack all node vectors into one matrix H and the whole layer, for every node at once, collapses to a single line:

H(next) = σ( D-1/2 (A + I) D-1/2 · H · W )

Read it left to right in plain words: take the normalized connection map (D-1/2(A+I)D-1/2), use it to mix each node's vector with its neighbors' (H), pass the result through a learnable transform (W), and squash it (σ). That is message, aggregate, and update, expressed as one matrix multiply. Every other GNN variant is a variation on how the middle "mixing" term is built.

In code, this is tiny In a library like PyTorch Geometric the entire layer above is one line, GCNConv(in, out). The framework precomputes the normalized connection map from your edge list, so you only ever pick the input and output vector sizes.

07Block diagram of the architecture

Zoom out from the arithmetic and a GNN has a clean, repeatable shape. Raw graph goes in on the left, a useful answer comes out on the right, and the middle is a stack of identical message-passing layers.

Input graph nodes + edges GNN LAYER 1 message aggregate update (repeat for all nodes) GNN LAYER 2 message aggregate update wider neighborhood Readout + prediction
Figure 2. The block diagram. Layers are stacked, and each one is the same three operations (message, aggregate, update) applied across the whole graph. Stacking more layers widens the neighborhood each node can "see." The final readout turns the polished node descriptions into the answer you actually want.

What the readout does

The middle of the network always produces one embedding per node. The readout is just the adapter that shapes those embeddings into the form your task needs:

If your task isThen the readout produces
Node-levelOne answer per node (for example, "is this user a bot?")
Edge-levelOne answer per pair (for example, "will these two people become friends?")
Graph-levelOne answer for the whole graph (for example, "is this molecule toxic?"), usually by pooling all node embeddings together

08Flowchart of one forward pass

Here is the same process drawn as a flowchart: the path a single graph takes from input to prediction, including the loop that repeats once per layer.

START: load the graph Give each node a starting feature vector Each node sends its vector to its neighbors (message) Combine incoming messages into one summary (aggregate) Blend summary with old vector to get a new vector (update) Done enough layers? Readout: turn vectors into the final prediction NO: run another layer YES
Figure 3. One forward pass. The three message-passing steps form a loop. The network repeats that loop once per layer, and only when the planned number of layers is finished does it hand off to the readout for the answer. Training then compares that answer to the truth and nudges the weights, but the flow above is what happens every single time the model "thinks."

09Watch it happen (interactive)

This is the graph from Figure 1, wired up with the exact rule from the worked example. Node A starts loud at 0.90; everyone else starts quiet at 0.10. Press Run one layer and watch the signal spread, hop by hop, into the parts of the graph that A cannot reach directly. The brighter a node glows, the higher its value.

Two extra controls let you feel the lessons from the rest of the page. The aggregation toggle swaps how each node combines its neighbors (the single choice that separates GCN, GraphSAGE, and GAT). And Run 12 layers lets you watch over-smoothing happen for real.

Message passing, one layer at a time

Each layer = every node blends half of itself with a summary of its neighbors.

Aggregation

Layer 0

Notice that after one layer only A's direct neighbors (B and D) move much. After two layers their neighbors move too, and so on. The signal can travel only one edge per layer, which is the single most useful intuition for choosing how many layers a GNN needs: roughly as many as the longest path information has to travel.

Now press Run 12 layers with mean aggregation. The original story (A is special) dissolves. Every node drifts toward a value set only by how connected it is, not by what it started with. The busy hubs B and E settle high, the rest settle low, and A is no longer distinguishable from its neighbors. That is over-smoothing: stack too many layers and every node's description blurs into the same structural mush, which is exactly why real GNNs stay shallow.

Try this Reset, switch aggregation to sum, and run a single layer. Values shoot up far faster, because sum does not divide by the number of neighbors, so well-connected nodes dominate. That sensitivity is the trade-off behind every aggregation choice.

10The architecture, layer by layer

Putting it all together, a standard GNN is built from four parts.

1. The input encoder

Every node arrives with some raw features. For a user that might be age and activity; for an atom it might be the element type. A small layer turns these raw features into the first set of vectors the message passing will work on.

2. The stack of message-passing layers

This is the heart. Each layer performs message, aggregate, update for every node, using learnable weights. Two layers see two hops out; three layers see three hops out. More layers reach further, but too many cause every node to blur into the same average, a failure mode called over-smoothing. Most practical GNNs use only two to four layers.

3. The readout (or pooling)

As shown in the block diagram, this converts the final node embeddings into the shape your task needs: per node, per edge, or one summary for the whole graph.

4. The loss and training loop

The prediction is compared against the correct answer, the error is measured, and the weights in every layer are nudged to make the next prediction better. This is ordinary gradient-based training; the only special part of a GNN is the message-passing machinery in the middle, not how it learns.

The mental model Input encoder gets everyone speaking the same language. The message-passing stack lets the gossip spread. The readout reads the room. Training tunes who listens to whom.

11Graph types in the wild

The tidy graph in our figures, undirected, unweighted, one kind of node, is the textbook case. Real graphs are usually richer, and a GNN can handle each twist with a small change to the message-passing rule rather than a new architecture.

TypeWhat changesEveryday example
DirectedEdges have a direction, so messages flow one way only"A follows B" on social media, or causal influence
WeightedEdges carry a strength, so some neighbors count moreRoad distance, or how tightly two parts are coupled
HeterogeneousSeveral node and edge types, each with its own ruleA graph of users, products, and reviews together
TemporalThe graph itself changes over timeA sensor network whose readings evolve every moment

That last row matters most for engineered systems. When the nodes are sensors and their readings stream in over time, you want a model that respects both the wiring (space) and the clock (time). These are spatio-temporal GNNs: a GNN handles the connections between sensors while a sequence model handles how each reading moves through time. It is the natural fit for anything that is "a network, measured continuously."

12The famous variants

Almost every well-known GNN differs in just one place: how it aggregates neighbor messages. Everything else is shared.

NameWhat makes it different, in plain words
GCNAverages neighbors, but weights them by how connected they are, so popular nodes do not shout over everyone else. The simple, reliable default.
GraphSAGESamples a handful of neighbors instead of using all of them, so it scales to enormous graphs like full social networks.
GATLets each node learn how much attention to pay to each neighbor, instead of treating them equally. Useful when some connections matter far more than others.

If you understand the message, aggregate, update loop, you already understand the shared skeleton of all three. The names just label different choices for the aggregate step.

13How powerful is a GNN?

A natural question: can a GNN tell any two different graphs apart? Surprisingly, no, and the reason is baked into message passing. Because every node only ever sees a summary of its neighbors, two graphs that produce the same summaries everywhere will look identical to the network, even if they are structurally different.

There is a classic hand-test that captures exactly this limit, called the Weisfeiler-Lehman (or "1-WL") color refinement test. It repeatedly relabels each node by combining its own label with the multiset of its neighbors' labels, which is precisely what a GNN layer does. The headline result: a standard message-passing GNN is at most as good at distinguishing graphs as that simple coloring test. If the test cannot separate two graphs, no amount of training will let a vanilla GNN separate them either.

This is why the aggregation choice from the demo matters more than it first appears. A sum aggregator keeps more information than a mean (mean throws away how many neighbors there were). The GIN architecture was designed around this insight: by using sum aggregation followed by a small neural network, it reaches the full power of the 1-WL test, the theoretical ceiling for this family. If your task hinges on counting or fine structure, this distinction is the difference between a model that can and cannot learn it.

14Inductive vs transductive

There are two very different jobs people ask a GNN to do, and confusing them is a common source of disappointment.

The difference is practical, not academic. A model trained on one batch of machines and then deployed on a new machine it has never seen is doing inductive learning. Variants like GraphSAGE were built specifically for this: by learning how to aggregate from a sampled neighborhood rather than memorizing fixed node positions, they transfer to unseen graphs. If your real goal is "train here, deploy elsewhere," you need an inductive setup from the start.

15A worked example from machine health

Here is where the graph view pays off in a setting far from social networks. Picture a simplified electric powertrain. The physical components and their sensors form a natural graph: each part is a node, and an edge means "this part directly influences that one." A fault does not stay put. It travels along these edges and shows up as a pattern of symptoms across several nodes, which is precisely the structure a GNN is built to read.

Battery Inverter Motor Bearing(wear starts here) currentsensor vibesensor tempsensor
Figure 4. A powertrain as a graph. Teal nodes are components, ochre nodes are sensors, gray lines are physical connections. The dashed rust arrows trace how a single bearing fault propagates: straight to the vibration sensor, into the motor current (so the current sensor sees a signature), and into added friction heat (so the temperature sensor drifts up). One root cause, three correlated symptoms.

Walk the fault through the graph. A bearing starts to wear. The most direct effect is mechanical, so the vibration sensor sees it first. But the added drag also changes the load the motor fights against, which leaves a fingerprint in the motor current, and the extra friction generates heat that the temperature sensor slowly picks up. A model that looked at each sensor in isolation would see three weak, separate anomalies. A GNN looks at the whole neighborhood at once and recognizes the joint pattern as one thing: bearing wear.

The same graph supports two complementary tasks straight out of the readout table:

And because a new vehicle has the same component graph as the ones you trained on, an inductive GNN can carry what it learned to hardware it has never measured. That combination, structure-aware diagnosis plus transfer to new units, is what makes the graph view more than a tidy diagram in health-monitoring work.

16Where it actually gets used

17Common misconceptions

Myth: more layers means a smarter model Past a handful of layers, every node's description blurs toward the same value (the over-smoothing you triggered in the demo). Depth in a GNN buys reach, not raw capacity. Most strong models are only two to four layers deep.
Myth: GNNs need huge graphs Some of the most successful GNNs work on tiny graphs. A drug molecule might have a few dozen atoms. The structure matters, not the size.
Myth: edges have to be physical wires An edge is any relationship you choose to encode: "is similar to," "was bought with," "influences." Defining the right edges is a modeling decision, and often the most important one you make.
Myth: a GNN is just a graph plus a normal neural net The defining ingredient is that the aggregation step does not care about the order of neighbors (it is permutation invariant). Relabel the nodes and the answer is unchanged. That property, not the presence of a graph, is what makes it a GNN.
Myth: it removes the need for good features A GNN mixes information, it does not invent it. If the starting node and edge features are uninformative, no amount of message passing will rescue the model. Feature design still matters.

18A short glossary

Node
A thing in the graph. A person, atom, sensor, or component.
Edge
A connection between two nodes, representing any relationship you choose.
Embedding
The vector of numbers a GNN learns to describe a node, shaped by its neighborhood.
Message passing
The core loop: send, aggregate, and update information between neighbors.
Aggregation
How a node combines its neighbors' messages into one summary (mean, sum, max, or attention).
Readout / pooling
The step that turns node embeddings into the final answer, per node, per edge, or for the whole graph.
Receptive field
How far across the graph a node can "see." It grows by one hop per layer.
Over-smoothing
The failure where too many layers make every node's embedding collapse toward the same value.
Transductive
Training and predicting within one fixed graph, with no unseen nodes.
Inductive
Generalizing to new nodes or entirely new graphs not present during training.
Homophily
The common tendency for connected nodes to be similar, which message passing exploits.
Permutation invariance
The property that relabeling or reordering nodes does not change the output. The signature of a true GNN.

The grid was always a convenient fiction. The real world is a graph, and a Graph Neural Network is simply the model honest enough to read it that way: by letting every node learn from the company it keeps.

By Majid Mazouchi.

Part 2 · A Working Introduction — architectures, limits & applications
Geometric Deep Learning · Monograph

Graph Neural Networks

How a neural network learns from data whose most important feature is who is connected to whom.

By Majid Mazouchi
The one-sentence version. A graph neural network (GNN) learns a representation of each node by repeatedly letting it exchange messages with its neighbours, so that after a few rounds every node's vector summarises not just itself but the structure around it. That single operation, message passing, is the whole field.

01 / MOTIVATIONWhy graphs need their own networks

Most successful neural networks assume a fixed, regular layout. Graphs break that assumption, and that is exactly why they need a different tool.

A convolutional network expects a grid: pixels with fixed neighbours, left, right, up, down. A recurrent or transformer model expects a sequence: a clean left-to-right order. Both lean hard on that regular structure.

An enormous amount of real data has neither. A molecule is atoms joined by bonds. A road network is intersections joined by streets. A knowledge graph is entities joined by relations. A power grid, a social network, a citation network, a fleet of components wired into subsystems: all of these are graphs, sets of nodes connected by edges, with no canonical ordering and no fixed number of neighbours.

Two properties make graphs awkward for ordinary networks, and a GNN is built to respect both:

  • Permutation invariance. Relabelling the nodes must not change the answer. A molecule is the same molecule whether you number its atoms 1-to-n or shuffle them. The model must not depend on the arbitrary order.
  • Variable structure. Different nodes have different numbers of neighbours, and different graphs have different numbers of nodes. The model must handle that gracefully, not assume a fixed-size input.
The mental model A CNN slides the same small filter over every patch of a grid. A GNN does the analogous thing on a graph: it applies the same small rule at every node, gathering information from whatever neighbours that node happens to have. The structure is irregular; the rule is shared.

02 / THE CENTRAL IDEAMessage passing, in plain words

Give every node a starting vector (its features: an atom's element, a user's profile, a sensor's reading). Then repeat one simple round, called a message-passing layer, as many times as you have layers:

  1. Each node sends a message to its neighbours, derived from its current vector.
  2. Each node aggregates the messages arriving from all of its neighbours into one summary. Because neighbours have no order, this aggregation must be a function that does not care about order: a sum, a mean, or a max.
  3. Each node updates its own vector by combining its old vector with that aggregated summary, usually through a small learnable transform and a nonlinearity.

That is the entire mechanism. The beautiful consequence is about reach. After one layer, a node knows about its immediate neighbours. After two layers, it knows about neighbours of neighbours, because its neighbours have themselves just absorbed their neighbours. In general, after k layers, every node's vector summarises its entire k-hop neighbourhood.

1 + 2. GATHER & AGGREGATE v ab cd m_v = AGG( msg(a), msg(b), msg(c), msg(d) ) then 3. UPDATE h_v′ = σ( W·[ h_v , m_v ] ) combine old self-vector with aggregated message v′ v now carries its neighbourhood
Figure 1. One message-passing layer at a single node. Messages from all neighbours are aggregated with an order-independent function, then combined with the node's own vector through a learnable, shared transform. The same rule runs at every node simultaneously.

03 / THE UPDATE RULEThe same idea, written down

The general message-passing form, which almost every GNN is a special case of, reads:

# h_v = vector of node v at the current layer; N(v) = its neighbours
mv = AGGREGATEu ∈ N(v) ( MESSAGE( hv, hu, evu ) )
hv′ = UPDATE( hv, mv )

where e_vu is an optional edge feature. The two design choices that distinguish one GNN from another are simply which aggregation and which update you pick. The most cited concrete instance is the graph convolutional network (GCN), which writes the whole layer as one tidy matrix equation:

H′ = σ( Â · H · W )

# H : stack of all node vectors (one row per node)
# W : learnable weight matrix, shared across all nodes
# Â : the normalised adjacency, Â = D^(-1/2) (A + I) D^(-1/2)

Read ·H as “replace each node's vector by a weighted average of itself and its neighbours.” The + I adds a self-loop so a node keeps some of its own signal, and the D^(-1/2) ... D^(-1/2) normalisation stops high-degree hub nodes from dominating. Multiply by W, apply a nonlinearity, and that is one GCN layer. Stack a few of them and you have a GNN.

Why the aggregator must be order-free A node's neighbours arrive as a set, not a list. If the aggregation were order-dependent (like a plain concatenation), shuffling the node labels would change the output and break permutation invariance. Sum, mean, and max all sidestep this, which is why they show up everywhere. The choice among them turns out to matter for how much the network can tell apart, a point that section 09 returns to.

04 / STACKING LAYERSDepth is reach

Each layer widens the horizon by one hop. This is the GNN analogue of a CNN's growing receptive field, and it is the single most useful intuition for choosing depth.

after 1 layer: 1-hop after 2 layers: 2-hop after 3 layers: 3-hop the red node's vector summarises everything inside the shaded region
Figure 2. Depth equals reach. Adding a layer expands each node's receptive field by one hop. This is why two or three layers is often enough on small-world graphs, and why going deeper is not automatically better (section 08).

05 / THE WORKHORSESGCN, GraphSAGE, GAT, GIN

Four architectures cover most of what you will meet. They differ only in the AGGREGATE and UPDATE choices from section 03.

GCN, the convolutional baseline

A degree-normalised weighted average of neighbours, the matrix form above. Simple, fast, and a strong default for node classification. Its weakness is that it must see the whole graph at once, which limits scale.

GraphSAGE, built to scale and to generalise

Instead of using the full neighbourhood, it samples a fixed number of neighbours and learns an aggregator (mean, pooling, or LSTM-style). Sampling makes it tractable on graphs with millions of nodes, and because it learns a rule rather than memorising node positions, it can be applied inductively to nodes and graphs it never saw in training.

GAT, attention over neighbours

Not all neighbours are equally relevant. A graph attention network learns a weight per edge, letting each node decide how much to listen to each neighbour, much like attention in transformers but restricted to the graph's edges. This helps when the graph mixes informative and noisy connections.

GIN, the expressiveness benchmark

The graph isomorphism network uses a sum aggregator plus a small MLP, a choice that, in theory, makes it as discriminative as the classical Weisfeiler-Lehman graph isomorphism test, the strongest a standard message-passing GNN can be. It is the reference point when the question is “can my GNN even tell these two structures apart?”

ModelAggregatorBest at
GCNnormalised meansimple, strong node-classification baseline
GraphSAGEsampled mean / poollarge graphs, inductive (unseen nodes)
GATattention-weightednoisy or unequal neighbours
GINsum + MLPmaximal structural expressiveness

06 / SEE IT WORKMessage passing, live

The whole mechanism is visible in one panel. Below is a real graph with two loosely connected communities. Two seed nodes are labelled (one blue, one orange); every other node starts grey, knowing nothing. Each layer runs one genuine round of normalised neighbour aggregation (the GCN operation, computed live, not animated by hand) and you watch the labels spread outward along the edges.

The two toggles expose the most important lesson in the field. With re-inject labels on, the seeds keep their identity each round and the labels diffuse into a clean classification: this is message passing solving a semi-supervised problem. Turn it off and keep adding layers, and you see the dark side: over-smoothing. Every node's value collapses toward the same grey, the variance across nodes drops to near zero, and the network can no longer tell any two nodes apart.

Message-passing sandbox

class A signal (+) class B signal (−) undecided (≈0) ringed = labelled seed
layers 3  |  feature variance --  |  classification accuracy --
What to try Leave re-inject on and slide layers from 0 upward: watch the two colours flood their communities and meet near the bridge edges, with accuracy climbing. Now turn re-inject off and push the layer count high: the variance readout falls toward zero and the whole graph fades to one grey. That collapse, identical vectors regardless of structure, is why most GNNs are only two or three layers deep.

07 / THREE LEVELSWhat you can actually predict

Once every node has a learned vector, you can ask questions at three scales. The GNN body is the same; only the head on top changes.

Node level

Classify or score individual nodes: which users are bots, which atoms are reactive sites, which components are degrading. The demo above is node classification. This is the most common GNN task and is usually done semi-supervised, labelling a few nodes and letting message passing carry that information to the rest.

Edge level

Predict whether an edge exists or what it means: recommendation (will this user like this item), knowledge-graph completion (does this relation hold), fault propagation (will a failure here reach there). You score a candidate edge from the vectors of its two endpoints.

Graph level

Summarise the whole graph into one prediction: is this molecule toxic, is this circuit faulty. Here you need a readout (also called pooling), an order-independent function, sum, mean, or max, that collapses all node vectors into a single graph vector before the final head. The order-independence requirement is the same one from section 03, now applied to the entire graph.

08 / PRACTICAL NOTESWhat actually goes wrong

GNNs are easy to wire up and have a handful of characteristic failure modes. These are where the research and the engineering pain both live.

Over-smoothing

The pitfall you just watched. Stack too many message-passing layers and every node's vector converges to the same value, because repeated averaging is a smoothing operation. The practical consequence: most GNNs peak at two to four layers. Mitigations: residual / skip connections, normalisation between layers, and designs that deliberately preserve a node's own signal.

Over-squashing

The opposite-feeling but related problem: information from an exponentially growing k-hop neighbourhood has to be squeezed through fixed-size node vectors and narrow bottleneck edges. Long-range dependencies get crushed. This is why some problems need graph rewiring or attention-style shortcuts rather than just more layers.

Limited expressiveness

Standard message passing is provably no more powerful than the Weisfeiler-Lehman test at distinguishing structures, so there exist genuinely different graphs a vanilla GNN simply cannot tell apart. If your task hinges on fine structural distinctions (counting cycles, say), you may need higher-order GNNs or added structural features.

Scale and sampling

Full-graph training does not fit in memory once graphs reach millions of nodes. Neighbour sampling (GraphSAGE) or subgraph / cluster sampling is what makes industrial-scale GNNs possible, at the cost of some variance in each update.

Heterophily

The averaging intuition assumes connected nodes are similar (homophily), which holds for citation and social graphs but not for all. When neighbours tend to be different (some fraud and interaction networks), plain GCN-style smoothing actively hurts, and you need models designed for heterophilous graphs.

KnobSensible defaultWhen to change it
Depth2–3 layersadd skips/norm before going deeper
Aggregatormean (GCN) or attention (GAT)sum (GIN) when structure must be distinguished
Self-loopsonalmost always keep them
Trainingfull-graph for small dataneighbour sampling for large graphs
Featuresnormalise node featuresadd structural / positional features if needed
Readout (graph task)mean or sum poolinghierarchical pooling for large graphs
When NOT to reach for a GNN If the relational structure carries little signal, or the graph is nearly fully connected (so everything neighbours everything), a GNN buys you little over a plain model on the node features. Reach for a GNN when who connects to whom is genuinely part of the answer.

09 / THE FAMILY TREEVariants you will hear about

Spectral vs spatial Two historical lineages. Spectral GNNs define convolution through the graph Laplacian's eigenbasis; spatial GNNs (the message-passing view used throughout this piece) work directly on neighbourhoods. GCN is the bridge that made the spatial view dominant.

Edge & relational R-GCN and friends handle typed edges, essential for knowledge graphs where “works-for” and “located-in” are different relations that should be processed differently.

Heterogeneous Real graphs often mix node types (users, items, sensors, faults). Heterogeneous GNNs keep separate transforms per type rather than pretending all nodes are alike.

Temporal / dynamic When the graph itself changes over time (a fleet's connectivity, a transaction stream), temporal GNNs fold a time dimension into message passing.

Graph transformers Attention over all nodes, with the graph injected as a structural or positional bias rather than a hard neighbourhood mask. These relax the locality of message passing and help with long-range dependencies and over-squashing.

Geometric deep learning The unifying umbrella: GNNs, CNNs, and transformers are all the same recipe (a shared local operation respecting a symmetry) specialised to grids, sequences, and graphs respectively. A useful frame for seeing why the field hangs together.

10 / IN PRACTICEWhere it earns its keep

GNNs are in production for recommendation, drug and materials discovery, fraud detection, traffic forecasting, chip design, and knowledge-graph reasoning. The common thread is that relationships, not just individual records, carry the signal. Two of these are worth drawing out, because they sit directly on the path of health monitoring and engineering knowledge work.

Knowledge graphs and reasoning

A knowledge graph of components, signals, faults, requirements, and documents is, literally, a graph. GNNs turn it into something you can compute over: link prediction suggests missing relations (this fault code plausibly connects to that subsystem), node classification tags entities, and learned embeddings power semantic retrieval. This is the relational backbone underneath a lot of agentic and retrieval tooling, where the graph supplies the structure that raw text search lacks.

Health monitoring and prognostics

  • Sensors as a graph. A set of sensors with physical or correlation-based relationships forms a graph; a GNN can detect anomalies and isolate faults by reasoning over how a deviation propagates between connected channels, rather than treating each channel alone.
  • Systems as a graph. Components wired into subsystems form a natural topology. Modelling fault propagation as message passing along that topology mirrors how failures actually cascade through a real system.
  • Fleet structure. Units that share parts, duty cycles, or operating conditions can be linked, letting evidence from one unit inform estimates on similar units, the kind of structured transfer that pure per-unit models miss.

The honest framing matches the PINN case: a GNN is not magic, but when the relationships between things are part of what you are trying to predict, it lets the model use that structure instead of throwing it away.

11 / REFERENCESWhere to go next

  1. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009). The graph neural network model. IEEE Transactions on Neural Networks, 20(1), 61–80. (the original GNN)
  2. Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations (ICLR). (GCN, the workhorse)
  3. Hamilton, W. L., Ying, R., & Leskovec, J. (2017). Inductive representation learning on large graphs. Advances in Neural Information Processing Systems (NeurIPS). (GraphSAGE)
  4. Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2018). Graph attention networks. ICLR. (GAT)
  5. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., & Dahl, G. E. (2017). Neural message passing for quantum chemistry. International Conference on Machine Learning (ICML). (the message-passing framework)
  6. Xu, K., Hu, W., Leskovec, J., & Jegelka, S. (2019). How powerful are graph neural networks? ICLR. (GIN and expressiveness)
  7. Li, Q., Han, Z., & Wu, X.-M. (2018). Deeper insights into graph convolutional networks for semi-supervised learning. AAAI Conference on Artificial Intelligence. (over-smoothing)
  8. Alon, U., & Yahav, E. (2021). On the bottleneck of graph neural networks and its practical implications. ICLR. (over-squashing)
  9. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021). A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1), 4–24. (survey)
  10. Hamilton, W. L. (2020). Graph Representation Learning. Synthesis Lectures on AI and Machine Learning, Morgan & Claypool. (the readable textbook)