A Plain-Language Field Guide · Part 3 of 7
← Part II · Limits, Machines, and Problem-SolvingPart III · A Multi-Agent Problem-Solver
If the human mind solves problems with a loop of cooperating mental acts, what does an AI framework that copies that loop look like? Part three turns the reasoning of the series into a working design, animated as it solves a real problem.
To build software that solves problems the way people do, the worst move is to build one all-knowing agent. The better move is to build a small team where the idea-makers are kept separate from the fact-checkers.
Part II ended on a picture: the human mind solves problems by being both a fast, careless idea generator and a slow, careful checker, with something in the middle deciding what to trust. A good agent framework copies that shape directly. It does not try to make one model brilliant. It splits the work into roles and, above all, keeps the part that proposes answers apart from the part that verifies them. That separation is where almost all of the reliability comes from, for the same reason it helps in people: a mind grading its own homework tends to pass itself.
Each mental act from the human loop becomes a role on the team. Here is the full cast, with its job in plain terms and the human faculty it stands in for.
None of these has to be a separate model. A role is a job with clear inputs and outputs, and several can share one underlying model with different instructions. What matters is that the jobs stay distinct. Tap each to see what it does.
The team lead. Holds the goal, decides who acts next, spends a fixed budget of steps wisely, and notices when the team is stuck.
Human counterpart: metacognition, the part of you that watches your own thinking and steers it. It reasons about the work, not the problem.
Turns a vague request into a sharp one. Clarifies the goal, decides what kind of problem it is, picks how to represent it, and splits it into smaller pieces.
Human counterpart: representation. As Part II showed, choosing how to see a problem is often half the solution.
Looks things up. Fetches relevant facts and past solved cases from the team's long-term memory, including ones that only match by deep structure.
Human counterpart: recall and analogy. It is how the team brings expertise to bear instead of starting from zero.
The idea machines. Several run at once with different angles, throwing out candidate answers, plans, or guesses. They are meant to be fast and often wrong.
Human counterpart: abduction, the creative leap. Nothing they say is trusted until it is checked, so boldness is safe here.
Works out consequences. Takes a candidate and turns it into a concrete, testable prediction, often by calling a tool like a calculator, simulator, or code runner.
Human counterpart: deduction. It converts "maybe X" into "if X, then we should see Y."
The fact-checker, deliberately a different agent from the proposers. Runs the test, scores the evidence, and actively tries to break the candidate rather than wave it through.
Human counterpart: critical thinking. This is the slow, exact check that gives the whole system its trustworthiness.
Acts on the world when thinking is not enough. Runs a probe, queries data, or tries something in a safe sandbox, and decides when to keep exploring versus commit.
Human counterpart: experiment and the explore-exploit tradeoff from Part II.
Files away what worked. After a solve, it writes the pattern back into long-term memory, indexed by structure, and records lessons from failures.
Human counterpart: deliberate practice. It is how the team's expertise grows over time.
The agents do not run in a fixed line. They share one workspace, and a coordinator decides who goes next. The shape is exactly the human problem-solving loop from Part II.
Everyone reads and writes to a shared notepad called the blackboard. It holds the current goal, the live guesses, and the evidence gathered so far. It is the team's working memory, and like human working memory it is deliberately small and temporary. The orchestrator looks at the blackboard, picks the next agent to act, and that agent posts its result back. Round and round, each pass narrowing things down. A finding from one turn becomes a starting fact for the next.
One more detail makes it behave like a real problem-solver: the blackboard holds beliefs loosely. If a later finding contradicts an earlier guess, that guess is simply withdrawn, with no crisis. This is the defeasible, non-monotonic reasoning from Part I, and it is what lets an agent change its mind gracefully when fresh evidence arrives. Step through a full turn below on a real-feeling problem.
Diagrams are clearer when they move. Below, the framework actually runs. Press play and watch control and data flow between the agents and the shared blackboard as the team solves one real problem from start to finish.
The problem is the same one from the last section: the nightly data pipeline failed. Each agent lights up when it acts. A pulse travels from the blackboard out to the agent when the orchestrator hands it control, and another travels back when the agent posts its result. Hypotheses appear, then turn green or red as the evidence comes in. The whole thing ends where it should, at a human gate.
Notice three things as it plays. Everything passes through the blackboard rather than agent-to-agent, which keeps the team's shared picture in one place. The proposers throw out a weak guess (the clock-drift one) that costs nothing because the verifier simply drops it. And the loop solves it in a single pass only because one cheap test split the hypotheses cleanly. When a test is not decisive, the team rounds the ring again, which is what the dashed loop is there to show.
The control loop says when the proposer thinks. It does not say how. A model that blurts its first answer reasons badly; a handful of strategies separate a guess from a worked solution.
They form a ladder of their own, from cheap and shallow to expensive and thorough. Pick one to see how it works and what it costs.
The through-line: every strategy is a different way to spend more computation when a problem earns it, by writing the steps down, by sampling more of them, by searching among them, or by looping with feedback from the world. They all rest on the same Part II insight, a fast fallible proposer made reliable by structure and checking. Match the strategy to the stakes: a one-line lookup does not need a tree search, and a safety-critical isolation does not deserve a single guess.
The team needs two kinds of memory, for the same reason you do: a small scratchpad for the problem in front of you, and a deep library of everything you have learned before.
The blackboard is the scratchpad. It is small on purpose, holding only what is relevant right now, and it is wiped when the job is done. Behind it sits the library: a large, lasting store of facts, skills, and past solved cases. When a new problem arrives, the retriever searches the library for anything that fits.
The most important design choice here is how the library is organised. Part II noted that experts file problems by their deep structure, not their surface look. The store should do the same, so that a new billing bug can pull up an old inventory bug because both are really the same kind of race condition. Indexing only by surface words means the useful past case sits right there and is never found.
Splitting memory into a scratchpad and a library is only the start. Cognitive science finds that people keep several distinct kinds of long-term memory, and a capable framework wants the same kinds, because they answer different questions. Tap each to see it.
The blackboard. Small, fast, and wiped when the job ends. Holds only what the team is thinking about right now.
In the framework: the shared workspace, kept inside the model's context window.
Specific past experiences. "Last Tuesday's run failed this way and here is what fixed it." Memories of particular events, with their context.
In the framework: the case library of past solved problems and full run traces.
General facts and concepts, stripped of when you learned them. "Pipelines have an extract, transform, and load stage."
In the framework: the knowledge graph and domain references the retriever queries.
Skills and how-to, the things you do without spelling them out. Riding a bike, or running a standard diagnostic.
In the framework: learned tools, reusable routines, and cached plans the agents can just call.
Knowledge baked into the model's weights (parametric) versus knowledge looked up from a store at the moment it is needed (external).
The design choice: bake in what is stable and universal, retrieve what is fresh, private, or specific. Retrieval is easier to update and to audit.
Once a model is deployed, its weights are frozen. That single fact decides the whole memory design: any knowledge that is new, changing, private, or specific to the task at hand has to live somewhere other than the weights.
In practice there are four stores, and what separates them most is how often each one can be written. The frozen weights are the fixed foundation. The other three are where all the live, updatable knowledge sits.
| Store | What it holds | Written | Read | Editable live? |
|---|---|---|---|---|
| Frozen weights | General language, broad world knowledge, baseline skill | Only in offline training | Every token | No |
| Short-term memory | The current task: goal, dialogue so far, intermediate results | Constantly, during the task | Constantly | Yes, but volatile |
| RAG store | Facts, documents, past cases (semantic and episodic) | Anytime, freely | When relevant to the query | Yes |
| Skills & workflows | Reusable tools and multi-step routines (procedural) | As new procedures are learned | When a task needs them | Yes |
Of the four, short-term memory is the easiest to overlook and the easiest to break. It is the running context the model sees on this step: the goal, what has happened so far, the latest retrieved facts, the partial answer. Everything the model "knows" in the moment that is not baked into its weights has to be loaded into this window. And the window is finite. So short-term memory is a budget, not a free space. When it fills, the system has to summarize and drop older detail to make room, which is exactly the forgetting work in the next section, applied first and most often right here. Manage it badly and the agent loses the thread mid-task; manage it well and a small window can carry a long, coherent piece of work.
Notice that these four are simply the concrete build of the memory types from two sections ago. Parametric knowledge is the frozen weights. Working memory is short-term memory. Semantic and episodic memory are the RAG store. Procedural memory is the skills-and-workflows library. The cognitive picture says what kinds of memory you need; this picture says where each one actually sits in a running system.
The learner does more than file notes. A framework can get better in several different ways, and they run on two very different clocks.
The fast clock turns during a single task: the system writes to its memory and reuses what it finds, without ever changing the model's weights. The slow clock turns across many tasks, offline, by actually retraining the model on what it has collected. Most of what people call "learning" in a live agent is the fast kind, and that is usually the right default because it is cheap, reversible, and easy to inspect. The slow kind is more powerful and more permanent, and more dangerous to get wrong.
Underneath, the classic learning regimes all have a place, distinguished by what signal they learn from.
| Kind of learning | Signal it uses | What changes | Where it lives |
|---|---|---|---|
| In-context | Whatever is on the blackboard and in retrieved memory | Nothing permanent; only the prompt and the stored cases | Runtime, every task (the cheap default) |
| Supervised | Labeled examples, or its own past successful traces | Model weights, toward imitating good answers | Offline, on collected data |
| Reinforcement | A reward for the outcome, often from the verifier | Model weights, toward higher-reward behaviour | Offline, using the verifier as the reward source |
| Unsupervised / self-supervised | Structure in unlabeled data, no answers given | Representations; discovered skills and groupings | Offline; also builds the structure-index for retrieval |
A tool or skill does not have to be a script. It can be a trained network that holds knowledge no one could write down by hand, and the learning method you choose decides what form that knowledge takes and what the tool is good for.
The skills store from earlier is more varied than a folder of code. Some entries are symbolic scripts, exact and verifiable, for procedures that are fully known. The rest are learned networks, for everything that is only known from data. The useful way to see it is that each learning method from the last section crystallizes one of the reasoning modes from Part II into a frozen, callable artifact.
Supervised learning makes predictive tools: it captures an input-to-output mapping, so the artifact answers "what will happen." In a vehicle-health setting these are forward dynamics models learned from logged trajectories, fast surrogates that stand in for an expensive physics simulation, remaining-useful-life regressors, and fault classifiers. Unsupervised and self-supervised learning make structural tools: with no labels, they capture the shape of the data, giving you the embeddings behind the retrieval index, anomaly and out-of-distribution detectors, and latent dynamics in the Koopman spirit, where a learned change of coordinates makes the dynamics nearly linear and hands you back something a controller can use. Reinforcement learning makes strategic tools, and that is the interesting case.
| Method | Form of knowledge | Tool it makes | Example | Used by |
|---|---|---|---|---|
| Supervised | Predictive (induction) | Dynamics model, surrogate, RUL regressor, classifier | Forecast winding temperature from logs | The reasoner, for model-based rollouts |
| Unsupervised | Structural (abstraction) | Embeddings, anomaly / OOD detector, latent dynamics | Flag an operating point as novel | The gate, and the retrieval index |
| Reinforcement | Strategic (experimentation) | Policy, value function, reward model, model-based controller | Learn which test isolates a fault fastest | The actor; value guides the search |
Supervised learning needs someone to have known the right answer and labeled it. Reinforcement learning needs only a reward, so it can acquire competence that nobody can demonstrate: a control law that exploits the dynamics in a way hand-tuned gains never find, or a behavior in a regime where the best action is not available in closed form. The form of knowledge it captures is neither declarative nor merely predictive. It is procedural and goal-relative, closed-loop know-how with credit assigned across a horizon. And it does not yield one artifact but several, each a different tool.
A policy is a learned skill, the network acting as a controller, for maneuvers too high-dimensional or nonlinear to script. A value function is a learned evaluator, an amortized answer to "how promising is this option," and it is quietly the most useful artifact for the agent itself, because it is a trainable version of the search heuristic from Part II and can rank hypotheses or steer a search the way a value network steers tree search. A reward model is a learned critic of quality where correctness is fuzzy. And model-based reinforcement learning ties the stack together: learn the dynamics, then plan or train a policy by rolling out through that model.
The cleanest example for this framework: deciding which cheap test to run next to isolate a fault fastest is a sequential decision under uncertainty with an obvious reward, information gained per cost. That is exactly the experimenter's job from the animated walkthrough, and it can be learned as an RL policy. The agent's exploration stops being a hand-written rule and becomes a trained skill. Reinforcement learning is, in effect, the explore step of the problem-solving loop compiled into a callable.
The reasoner does not need a learned network for everything. It routes by the nature of the subproblem. Try it: pick what you are facing, and see which kind of tool fits.
This is the rule that keeps learned tools safe inside the framework. A trained network is exactly the fast, confident, sometimes-wrong proposer the whole series has warned about, so it is treated the same way. Every learned tool ships with calibrated uncertainty, from an ensemble or a Bayesian head, and an out-of-distribution check. When the anomaly detector says the situation has left the region the tool was trained on, the agent retracts its trust in that tool and falls back to deliberate reasoning or a human. That is the defeasible behavior from Part I applied to a learned component. Its output still passes the verifier like any other proposal. And reinforcement learning carries its own version of confirmation bias, called reward hacking, where the policy games a flawed reward, which is caught the same way the bias-monitors catch the others: by trying to break the reward, not just satisfy it.
A memory that only ever grows becomes slow, stale, and unsafe. Forgetting is not a bug to be fixed. It is a feature that keeps memory useful, and people rely on it just as much.
Left unchecked, a store fills with duplicates, outdated facts, dead ends, and things that should never have been kept. Retrieval gets noisier, costs climb, and the system starts acting on beliefs that are no longer true. So a real framework needs deliberate ways to let go. Tap each.
When the blackboard overflows the context window, old detail is compressed into a short summary and the raw text is dropped.
Why: working memory is bounded, exactly like ours. Summarizing keeps the gist while freeing room.
Entries that are old and rarely retrieved lose priority and eventually drop out of easy reach.
Why: what you have not needed in a long time is usually safe to let fade, which is how human forgetting works too.
When new evidence contradicts a stored fact, the old fact is retracted rather than kept alongside the new one.
Why: this is the defeasible belief revision from Part I. Holding both makes the system incoherent.
Many near-identical cases are merged into one cleaner, more general pattern.
Why: it mirrors how sleep is thought to compress memories, and it keeps the library sharp instead of bloated.
Cases that never help on later problems are scored down and removed.
Why: a store should keep what earns its place, not everything that ever happened.
Some things must be removed on purpose: private data, a right-to-be-forgotten request, or a poisoned or wrong memory.
Why: this is easy in an external store and genuinely hard once knowledge is baked into weights, where it becomes machine unlearning.
What makes the framework feel human is not the agents but the supervisor over them, and a set of small watchdogs whose only job is to catch the classic ways problem-solving goes wrong.
The orchestrator does two human things. First, it spends a budget. Just as you decide where to put your limited attention, it decides how many tool calls and how much compute each part of the problem deserves, and stops chasing a dead end. Second, it watches for trouble. The failure modes from Part II are predictable, which means they can be detected, so the best move is to build a small monitor for each one.
A fixation monitor notices the same failing approach being retried and forces a switch. A premature-closure monitor refuses to let the team commit until at least one rival idea has been weighed. A confirmation monitor makes the experimenter run a test that could disprove the favourite, not just confirm it. A local-optimum monitor spots the team polishing a mediocre answer and triggers a fresh start from a new angle. These watchdogs are cheap and they prevent the most common and most expensive mistakes. Try switching one on and off below.
The team has a favourite explanation that fits the evidence. Should it ship the fix, or first try to prove itself wrong? Toggle the monitor, then run.
A system that can act in the world needs two safety pieces that have nothing to do with intelligence and everything to do with not causing harm: a person at the important door, and a safe room for experiments.
The first guardrail is a human-in-the-loop gate. Before the team does anything costly or hard to undo, and before the final answer ships, a person approves it. This is the same idea as the trusted checker from Part II, now with a human as the last verifier: let the agents propose freely, but put a gate in front of consequences. Reversible, low-stakes steps can flow without it; the gate is reserved for the moves that matter.
The second guardrail is the sandbox. When the experimenter runs code or probes a system, it does so in an isolated environment that cannot touch anything important. The key rule is that the thing controlling the sandbox sits outside it. An agent must never be able to reach out and modify the cage it is running in, or the cage is not a cage.
There is no single sandbox, but a spectrum. Stronger isolation costs speed, so you match the level to how much you trust the code being run. A trusted snippet you wrote needs far less caging than code an agent generated on the fly.
Runs a cut-down, safe subset of the language inside your own process. Lightest and fastest, weakest isolation.
Use when: the code is largely trusted and you only need to block a few dangerous operations.
Runs code inside a WebAssembly box with no direct access to the host. Moderate isolation, still quite fast.
Use when: you want a clean, portable boundary without standing up a full machine.
An operating-system-level container hardened with a syscall filter that sits between the code and the real kernel. Strong, and the common choice.
Use when: running untrusted, agent-generated code at scale.
A tiny, fast-booting virtual machine giving near-hardware isolation. Strongest, heaviest.
Use when: the code is fully untrusted or possibly hostile, and a breakout would be serious.
A different idea: instead of isolating code, give the experimenter a model of the world (a physics simulator, or a staging copy of the system) to act in.
Use when: real-world experiments are slow, costly, or irreversible. Test on the copy, not the original.
The human comparison is a guide, not a law. Three places where it breaks down matter a great deal when you actually build one of these.
First, the "fast" agents are not fast. In a person, the quick intuitive guess is nearly free. In this framework, every agent is a slow, expensive model call, so a design with many roles can be far slower and costlier than a single well-instructed agent. The mind's cheap-proposal, expensive-check asymmetry is something you have to engineer toward, not something you get for free.
Second, more agents is not more intelligence. Each new role adds coordination overhead and new ways for the team to misunderstand each other. A tight team of three often beats a sprawling team of twelve. Add a role only when you can name the specific failure it fixes.
Third, and most important, the whole thing is only as good as its verifier. The reliability story rests entirely on having a way to check answers against ground truth: a test suite, a simulator, a formal checker, the real environment. Where that exists, this architecture is genuinely powerful. Where checking is itself fuzzy and subjective, the propose-and-verify backbone weakens, and you are back to trusting fluent text, which is the exact trap the whole series has warned about.
This part builds on the foundations in Parts I and II. The works below cover the agent and cognitive architectures behind the design.