A Plain-Language Field Guide · Part 7 of 7
← Part VI · The Dynamics of ThinkingPart VII · Understanding and Comprehension
All the earlier parts built the machinery of thought. This one asks what it is for. Understanding is the model a mind builds so the world becomes predictable from the inside, and comprehension is how that model gets built. We follow it from words to meaning, ask what a model really is, why prediction is not enough, how it breaks and how we test it, and whether a machine can do it, or we it.
Every part so far built the machinery of thought: inference, the agent, the society, the handling of thin knowledge, and the dynamics of the motion itself. This part asks what all of it is for. The answer is understanding.
Understanding is not having information. A system can store a fact and hand it back without understanding it at all, the way a phone book holds numbers. Understanding is having a model of something that captures its structure well enough that you can leave the given behind: predict what it will do next, explain why it does it, apply it to a case you were never shown, and answer what would happen if it were different. The phone book cannot tell you what a number would be if the city added an area code; someone who understands the numbering scheme can.
It helps to separate three things we lump together. Knowing that a bearing fails at a certain temperature is a fact. Knowing why, the mechanism of heat and wear and lost lubrication, is explanatory understanding. Knowing how to spot it on a machine you have never serviced is procedural understanding. Facts can be memorized; the why and the how have to be built, and they are what we mean by understanding in the deep sense.
There is also a shallow kind that imitates the deep kind. A student who can recite a definition, or a model that completes a familiar phrase, looks like it understands until you move the problem an inch. The tell is not how fluent the answer sounds but whether the model behind it holds up when the situation changes. The rest of this part follows understanding from the outside in: how comprehension builds a model from an input, why it runs on what you already know, how it settles, how it breaks and repairs, how we can tell it is real, and whether a machine can do it.
Comprehension is understanding aimed at an input, most often language. It is the work of turning marks on a page into a model of what they are about, and it happens in layers.
The cleanest account comes from text comprehension. At the bottom is the surface, the words themselves. Above it is the textbase, the bare propositions the words encode, who did what to whom. Above that is the situation model, an integrated picture of the state of affairs the text describes, fused with everything you already know about such situations. Only the top layer is understanding. The words are the doorway; the situation model is the room you walk into. Build the layers on a tiny example.
Notice how much of the final model was never stated. The text never said she wanted ice cream or that the truck sold it; you supplied both from prior knowledge without noticing. Comprehension is less like recording and more like construction, and most of the material comes from the builder, not the blueprint.
If most of a model comes from the builder, then understanding is mostly a matter of what the builder already holds. The most reliable predictor of whether you will understand something is how much related structure you already have.
Knowledge is organized into schemas, patterns of what usually goes with what. A new input is understood by being slotted into an activated schema, which fills the gaps and says where to look next. This is why the same input lands so differently for an expert and a novice. Shown a handful of symbols, the novice holds each as a separate item; the expert sees a few meaningful chunks and has room to spare. The symbols are identical; the understanding is not, because the prior structure is not.
Connection also works sideways, through analogy. We understand a new domain by mapping it onto one we already know: the atom as a little solar system, the circuit as water in pipes, the immune system as a defending army. The map is never perfect and it can mislead, but it is how a foothold is first found. Understanding, in the end, is less a possession than a position in a web. A fact you can reach from many directions is understood; one that floats unconnected is merely stored.
There is a sharper way to say what a model is. You understand a system when you can run it. Given a situation, you can turn the crank in your head and say what happens next, the way you can trace a fault through a fleet or feel how a control loop will respond before you ever build it.
This is the difference between a black box and a mechanism. A black box maps inputs to outputs by association: you have seen this input go with that output, so you expect it again. A mechanism tells you why, so it can be run on an input you have never seen and a situation that has never occurred. The black box holds a table; the mechanism holds a model, and only a model can be simulated forward. Craik put it plainly almost a century ago: a mind that understands carries a small-scale model of reality and runs it to try alternatives before committing to one.
The model need not carry numbers. Much of expert understanding is qualitative. You do not compute the bearing temperature; you know that more load raises heat, heat thins the lubricating film, a thinner film raises friction, and friction raises heat again, a loop that can run away. That chain is a runnable model with no arithmetic in it, and running it is what lets you predict a failure you have never measured. Set the conditions and run it.
If a model can be run on cases you never saw, it must be smaller than the list of all those cases. That is the next idea: understanding is compression. A model that captures the structure of something is the shortest description that can regenerate it.
Hold two ways of knowing a set of values side by side. One stores every pair, a lookup table that grows with the data and is silent on anything it has not stored. The other finds the rule that generates them, a short description that fits on one line and answers for inputs it has never seen. The table is memorization; the rule is understanding, and the gap between them is exactly compression. The rule is shorter than the data because it has thrown away the noise and kept the structure, which is the same act as having understood it. Query a value the data never contained.
A quality test comes with this picture. A good explanation, Deutsch argued, is hard to vary: every part does work, so you cannot change a piece without breaking its fit to reality. A bad explanation is easy to vary, its parts slide around and it still seems to explain anything, which is why it explains nothing. A short, hard-to-vary description is the signature of real understanding; a long, freely adjustable one is the signature of a story bolted onto memorized facts.
This is the same lesson generalization teaches in learning. A model that merely memorizes its training data is long and brittle and fails the moment it steps off that data; one that finds the compact underlying rule is short and transfers. The quantity that predicts whether a learner will generalize is, loosely, how much it compresses rather than stores, which is why understanding and generalization are two names for one thing.
It is tempting to equate the two: if a model predicts well, surely it understands. It does not follow, and seeing why is the cleanest correction this part can offer.
A system can predict beautifully from correlation alone, with no grasp of what causes what. The rooster predicts the dawn; the falling barometer predicts the storm. Both are reliable and neither understands anything, because neither survives an intervention: silence the rooster and the sun still rises, hold the barometer needle in place and the storm still comes. Prediction rides the surface correlations; understanding reaches the mechanism beneath, and the distance between them is the distance between watching and grasping. Put the difference to the test.
This is the ladder of causation from Part I, met again. The bottom rung is seeing, predicting from observed association. The middle rung is doing, predicting what follows when you intervene. The top rung is imagining, answering what would have happened had things been different. A predictor lives on the bottom rung; understanding requires the rungs above, where you can act on the system and reason counterfactually about it. A model that cannot say what its prediction would be under an intervention has not understood, however accurate it is.
This sharpens the question that closes the part. High accuracy at predicting what comes next is bottom-rung competence, and it leaves open whether a system would handle the intervention and the counterfactual, which is where understanding actually lives. It is why "it predicts well" can never settle the matter on its own.
Borrow the lens from Part VI and the whole process snaps into focus. Comprehension is a settling. The mind relaxes toward the interpretation that satisfies the most of its constraints at once, and the moment it understands is the moment that relaxation reaches a stable resting point.
An input arrives underdetermined, with more than one possible reading. Each piece of context pushes for some interpretations and against others, the way a word's neighbours decide which of its meanings is meant. These pushes are constraints, and the mind settles them all together, letting the readings that support each other rise and the ones that conflict fall, until one coherent picture wins. That is a constraint-satisfaction network relaxing into an attractor, the very dynamics of Part VI, now doing the work of meaning. Pick a context and watch the readings settle.
This explains several familiar things at once. Ambiguity feels effortless because the settling is fast and you only ever see the winner, never the competition. A coherent passage is one whose constraints have a clear joint solution, a deep single valley; an incoherent one has none, so the reading wanders and never settles, which is the felt experience of confusion. And context works before you notice it, biasing the landscape so the right reading is already winning by the time it reaches awareness.
Turn the last idea around and you get a deep account of how a mind understands at all. If understanding is a runnable model, the mind may run it constantly, predicting its own incoming signals and learning only from where it is wrong. That is the predictive-processing view, and it speaks the language of this series directly: error as the driving signal.
The proposal is that the brain is not a passive reader of the world but a prediction machine. At each moment it uses its model to predict what it is about to sense, compares that to what arrives, and passes only the mismatch, the prediction error, onward. What you perceive is mostly the prediction; what you learn from is the error. Understanding, on this view, is having a generative model good enough that the errors stay small, and comprehension is the act of fitting that model to the input until the surprise drops. Watch the loop chase a moving world.
Stated this way, almost everything in this part is one mechanism seen from different sides. The situation model is the generative model. Comprehension as settling is that model relaxing until prediction error is minimized. The monitor that fires when something does not add up is a spike of error. Insight is a sudden model revision that collapses a standing error. And learning is what happens when the world keeps surprising you until the model finally changes to stop being surprised. One loop, predict, compare, update, runs beneath all of it.
That loop is a controller, which is why it sits so naturally at the close of a series that read thinking as dynamics. The model supplies a prediction, the world supplies a measurement, the error between them drives an update, and the system settles when the error is small, the same shape as a regulator holding a setpoint. A mind that understands its world is one whose prediction error stays low as the world moves, and curiosity is the policy of seeking out the errors worth reducing.
A model under construction can go wrong, and a healthy comprehender notices. The quiet faculty that watches the building and raises a flag when it stops hanging together is comprehension monitoring, and it is the verifier of Part III wearing another hat.
As you read or listen you are continuously checking that the model coheres, that each new piece fits and that the predictions it makes keep coming true. When a piece does not fit, the monitor fires, often as nothing more than a faint sense that something is off, and construction pauses for repair. Watch it happen in a sentence built to fool the monitor.
Repair comes in two sizes. When the new fact can be absorbed with a small adjustment, you assimilate it and read on. When it cannot, the model itself has to be rebuilt around it, and that larger move is accommodation. The sudden, satisfying version of accommodation, where a confusing whole reorganizes all at once into a clear one, is what we call insight, and dynamically it is the bifurcation from Part VI, the trajectory snapping from one basin to another.
The dangerous failure is the opposite of confusion. A fluent, familiar passage can sail past the monitor and leave you feeling you understood when you built almost nothing, the illusion of understanding that fluency produces. Feeling that it made sense is not the same as holding a model that works, which is why a feeling of understanding can never be the test; only moving the problem can, as the transfer section will show.
The monitor that should catch a missing model can be fooled, and there is a clean experiment that fools almost everyone. Ask people how well they understand an everyday thing, a zip, a flush toilet, a bicycle. They rate it high. Ask them to explain exactly how it works. The rating collapses.
This is the illusion of explanatory depth, and it is not stupidity but the normal economy of the mind. We carry a sense of understanding that runs on familiarity and on how easily an answer seems available, not on actually holding the mechanism. Because we can usually get the detail when we truly need it, from a glance or a quick lookup, we feel as though we already have it. The feeling and the model come apart, and the one thing that reliably reveals the gap is being made to produce the mechanism out loud. Try it on yourself.
The move that exposes the illusion also cures it. Forcing yourself to explain, to a student, a blank page, or a rubber duck, turns the felt sense into an actual attempt to run the model, and the gaps appear wherever the run stalls. This is the self-explanation effect: learners who explain each step to themselves understand far more than those who only read, because explaining is the only way to find out whether a model is there to run.
Prior knowledge does most of the work of understanding, and that is exactly why it can also block it. The same web that lets you absorb a new fact quickly can hold a wrong model so firmly that the right one cannot get in.
People do not arrive empty. They come with intuitive theories built from a lifetime of experience, and many are wrong in deep, systematic ways. It feels as if heavier things fall faster, as if a moving object needs a continuous push to keep moving, as if the seasons come from the Earth's distance to the sun. These are not random slips; they are coherent, useful-enough models that ordinary life rarely contradicts. A new fact that conflicts with one does not simply overwrite it; it gets bent to fit, explained away, or memorized for the test and quarantined from the intuition that still runs the rest of the time. See how stubborn a wrong model can be.
Correcting such a model is not adding a fact but conceptual change, and it has the shape Part VI described. The wrong model sits in a deep, stable basin; small contrary evidence cannot dislodge it, and the learner shows the hysteresis of belief, needing far more to leave the old view than was ever required to hold it. Real understanding arrives only when the whole structure is rebuilt around the new idea, the hard, sudden reorganization that the repair section called accommodation. Short of that you get the worst case of all: a learner who can state the correct answer and still reasons with the old model the instant the question is dressed differently.
Understanding is not on or off; it comes in levels, and naming them turns the vague shallow-to-deep axis into a ladder you can locate yourself on.
A useful ordering runs from the thinnest grasp to the fullest. At the bottom is recall, returning the thing as given. Above it is explanation, saying it in your own words and giving the why. Above that is application, using it on a new case. Higher still is transfer, carrying it to a domain that shares only the deep structure. And at the top is the level that proves all the others, being able to teach it and to critique it, to find where it breaks and what it assumes. Each rung subsumes the ones below: you cannot truly teach what you cannot transfer, or transfer what you cannot explain. Climb it.
The ladder explains why so much that passes for understanding is the bottom rung wearing the clothes of the top. A student who recites a derivation, or a model that completes a proof it has seen, sits on recall while sounding like explanation. The way up is not more exposure but more demand: ask for the why, then the new case, then the far case, then the lesson taught to someone else. Understanding is built by being required to operate one rung higher than feels comfortable.
Because fluent recall can masquerade as understanding, you cannot test understanding by asking for the thing back. You have to move the situation and see if the model still works. That movement is transfer, and it is the only honest test.
A handful of probes separate a model from a memory. Try each.
Transfer comes in distances. Near transfer, to a case much like the one you learned on, is easy and proves little. Far transfer, to a situation that shares only the deep structure, is hard and is the real evidence, because only the structure survives the trip and only an understood structure can make it.
There is a mirror to the question that closes this part. We have asked whether the machine understands. The reverse is just as pressing: do we understand the machine? We now build systems we can predict only loosely and explain hardly at all, and the gap is the same one this whole part has been about.
A large model is, from the outside, the very thing this part warned against, a system known mostly by correlation. We have watched what it does across millions of inputs and learned to expect certain outputs, which is bottom-rung, predict-from-association knowledge of it. We can rarely say why it produced a given answer, what internal mechanism generated it, or what it would do under an intervention we have not tried. We hold a table of its behaviour, not a model of its workings, which is to say we do not yet understand it in the deep sense. Open the box.
The effort to close that gap is mechanistic interpretability, and it is understanding aimed at a machine rather than a text. It tries to recover the model inside the model: the internal features a network represents, the circuits that compute with them, the mechanisms behind a behaviour, so that prediction can be replaced by explanation. It is the same climb from the bottom rung to the higher ones, now applied to an artefact, and it matters because a system you can only predict you cannot fully trust, audit, or correct, which is exactly the gate-holder's problem from earlier in the series.
Notice the symmetry. The two open questions of this part are mirror images: does the machine build a model of the world, and can we build a model of the machine. Both ask whether there is structure under the surface behaviour or only correlation, and both are answered the same way, not by how well one side predicts the other but by whether the mechanism can be recovered and run. Understanding, in both directions, is the recovery of the mechanism.
All of this lets us ask the question of the moment with sharper tools. When a large model answers well, is it understanding, or only producing the surface that understanding would have produced?
Two serious answers compete, and it is worth hearing each at its strongest rather than its weakest.
This part offers a way to cut through some of it. Set recall aside and apply the test that works on people: transfer. Not whether the system can return what it was trained on, but whether it holds up when the structure is kept and the surface is moved, whether it answers the counterfactual, whether the model survives a change of situation. By that measure understanding is no more all-or-nothing in a machine than in a student, and the honest answer to the headline question is a quantity, not a yes or a no.
Sources behind Part VII, on comprehension and situation models, schema, expertise and analogy, mental models and compression, causation and prediction, predictive processing, the illusion of explanatory depth, conceptual change, levels of understanding, and machine interpretability.