Why one large language model is rarely enough — roles, memory, protocols, orchestration patterns, failure modes, costs, evaluation, and governance, explained like an engineering team.
A single chatbot answers questions. A team of specialized agents — each with one clear job, connected by feedback and supervised by an orchestrator — can complete entire workflows.
This is the central message of the IBM Technology video. Humans solved the problem of complex work long ago: we split it into roles, give each person a clear responsibility, and add review and management on top. Agentic AI borrows the same playbook.
In simple words: a chatbot responds to a prompt and stops. An agent has a goal. It makes a plan, calls tools (search, databases, code execution, APIs), remembers context, and takes multiple steps until the goal is reached — or until it decides it cannot reach it.
IBM's working definition: an AI agent is a system that can autonomously perform tasks by designing its own workflow using the tools available to it. The keyword is autonomously — it decides the next step itself instead of waiting for you to type the next prompt.
| Plain chatbot | AI agent |
|---|---|
| Reacts to one prompt at a time | Pursues a goal over many steps |
| Only generates text | Plans, calls tools, takes actions |
| No memory of the task state | Tracks progress, context, and intermediate results |
| You drive every step | It drives; you set the goal and the limits |
A single model asked to "do everything" tends to do everything poorly. Planning, coding, retrieving facts, and reviewing quality are different cognitive jobs. When one prompt has to cover all of them, the instructions blur and the model cuts corners.
Watch the same task — "Write a verified summary of failure rates from our fleet database" — handled both ways:
The video describes six recurring roles. Each maps cleanly onto a familiar human job:
Breaks a big goal into small, ordered steps. "Build me a mobile app" becomes: requirements → UI design → code → test → review → deploy.
Human equivalent: System architectProduces the actual output — the code, the report, the document. It executes the plan rather than inventing one.
Human equivalent: EngineerKnows how to call external systems: databases, APIs, web search, code runners, file systems, enterprise software.
Human equivalent: Tools specialistFetches missing information from documents, knowledge bases, the web, or examples — so the doer doesn't have to guess.
Human equivalent: ResearcherChecks the output against the requirements, finds mistakes, and suggests improvements. It reviews; it never rewrites.
Human equivalent: ReviewerWatches the whole workflow, routes work between agents, decides when the result is good enough, and stops the loop.
Human equivalent: Project managerOnce the roles exist, quality comes from four design levers:
| Lever | What it means | Practical note |
|---|---|---|
| 1. Prompting | Each agent gets a clear, narrow job description. | Write prompts like job descriptions: responsibility, inputs, outputs, and explicit "do not" rules ("the critic never edits the draft"). |
| 2. Model selection | Different models for different jobs. | Reasoning-heavy roles (planner, critic) deserve the strongest model. Routing and formatting can run on small, cheap models. |
| 3. Model tuning | Adjusting models so they behave consistently in a domain. | For enterprise work, consistency beats brilliance. Fine-tuning or strict configuration reduces day-to-day variance. |
| 4. Context | The right information at the right time. | Too little context → hallucination. Too much irrelevant context → confusion. Curate what each agent sees. |
"Context" in Section 05 is really three different memories with three different lifetimes. Confusing them is a common design error.
| Memory type | Lifetime | What it holds | Who needs it |
|---|---|---|---|
| Short-term (scratchpad) | One task | The current plan, intermediate results, tool outputs, draft versions. Lives in the context window. | Every agent, but only its own slice — the doer's scratchpad shouldn't leak into the critic's. |
| Shared (blackboard) | One workflow | The common task state all agents read and write: the accepted plan, the evidence list, the current verdict. | The supervisor owns it; agents get read access plus write access to their own fields. |
| Long-term / episodic | Across tasks | Past solutions, user preferences, domain facts — usually a vector store or database queried on demand. | The learner/researcher primarily; others through it. |
Two open protocols are emerging as the standard wiring for agent systems. They solve different problems and complement each other:
MCP, introduced by Anthropic, standardizes how an agent connects to tools, data sources, and services. Instead of writing custom glue code for every database, file system, or API, you run an MCP server that exposes them in one uniform format any MCP-capable agent can use. Think of it as USB-C for tools: one connector shape, many devices.
A2A, introduced by Google and now hosted by the Linux Foundation, standardizes how agents discover and talk to each other — across vendors, frameworks, and companies. Each agent publishes an "agent card" describing what it can do; other agents can then delegate tasks to it, track progress, and receive results, without sharing internal memory or prompts.
| MCP | A2A | |
|---|---|---|
| Connects | An agent to tools and data | An agent to other agents |
| Analogy | USB-C port for capabilities | A common business language between teams |
| Typical use | "Query this database," "read this file," "call this API" | "Delegate this sub-task to the specialist agent and await its result" |
| Inside the team metaphor | How the tool operator holds its tools | How the supervisor assigns work to teammates |
"Multi-agent" is not one architecture. Four patterns cover most real systems:
| Pattern | Shape | Strengths | Weaknesses | Pick when… |
|---|---|---|---|---|
| Sequential pipeline | A → B → C → D, fixed order | Simple, predictable, easy to debug and audit | Rigid; can't adapt when a step fails or the task varies | The workflow is genuinely fixed (e.g., extract → draft → verify → format) |
| Supervisor (hub-and-spoke) | One orchestrator routes all work to specialists | Central control, clear stop conditions, easiest governance | Supervisor is a bottleneck and single point of failure; its routing quality caps the system | You need auditability and bounded behavior — the default for enterprise |
| Hierarchical | Supervisors of supervisors; teams within teams | Scales to large problems; each sub-team stays simple | More layers = more latency, more cost, harder tracing | The task naturally decomposes into sub-projects (e.g., per-subsystem documentation) |
| Peer-to-peer (swarm) | Agents hand off to each other directly, no boss | Flexible, resilient, no central bottleneck | Hardest to bound and audit; loops and drift are common | Exploratory tasks where adaptability beats predictability — rarely first choice in regulated domains |
A good multi-agent system does not generate one answer and stop. It drafts, reviews, revises, and only then delivers. Agents can use feedback from other agents or from humans to improve their output — this reflection step is where most of the quality gain comes from.
But the feedback loop is also the riskiest component:
Every one of these will eventually appear in a real deployment. Knowing the symptom in advance is half the cure:
| Failure mode | Symptom you'll see | Mitigation |
|---|---|---|
| Infinite loop | Critic and doer ping-pong revisions forever; latency and cost climb with no convergence. | Hard iteration cap; supervisor escalates to a human after N rounds; require the critic to output a bounded verdict (accept / revise / escalate), never open-ended chat. |
| Sycophantic consensus | Agents quickly "agree" — the critic rubber-stamps whatever the doer wrote. Output is fluent and wrong. | Give the critic ground-truth evidence, not the doer's reasoning; use a different model for the critic; reward finding issues ("list at least the 3 weakest claims") rather than approval. |
| Context poisoning | One early wrong "fact" propagates: every downstream agent repeats and builds on it. | Tag every claim with its source; let deterministic extraction outrank LLM inference; re-verify key facts at the final gate instead of trusting upstream agents. |
| Tool overuse | An agent calls search or a database dozens of times per task; bills spike, rate limits trip. | Per-agent tool budgets; cache repeated queries; make the agent state why it needs each call; alert on outlier call counts. |
| Role bleed | The critic starts rewriting the draft; the planner starts executing; outputs become untraceable. | Explicit "do not" rules in every prompt; structured output schemas that physically can't carry the wrong content (a critic schema has no "revised draft" field). |
| Lost context / overflow | On long tasks, early decisions vanish from the window; agents contradict the original plan. | Blackboard memory for durable state; summarize-and-pin the accepted plan; don't rely on raw chat history as the system of record. |
| Silent partial failure | A tool call fails, the agent improvises a plausible value, and nobody notices. | Fail loudly: tool errors must surface to the supervisor as structured events, never be paraphrased away; verifier checks that every claimed tool result has a matching log entry. |
Put together, a typical multi-agent workflow looks like this:
Now step through a live example — "Summarize last quarter's top three warranty failure causes for the fleet" — and watch the message at each hop:
Notice two things. First, the supervisor sits above the loop, not inside it — it referees but does not produce content. Second, the critic sits between the doer and the final answer: nothing ships unreviewed.
Multi-agent systems multiply token usage. Every hop re-sends context, and every revision round repeats the doer + critic pair. A task that costs one unit as a single prompt commonly costs 3–10× as an agent team — more if context is broadcast instead of routed.
| Driver | Why it multiplies cost | Countermeasure |
|---|---|---|
| Context re-sending | Each agent call resends shared background in its prompt. | Blackboard memory + per-role context slices; prompt caching where the platform supports it. |
| Revision rounds | Each loop ≈ one doer call + one critic call. | Cap at 2–3 rounds; require the critic to list all issues in one pass instead of one per round. |
| Oversized models everywhere | Using the flagship model for routing and formatting wastes most of its capability. | Tier the models: flagship for planner/critic, mid-tier for the doer, small for routing. |
| Latency stacking | Sequential hops add up — five 4-second calls is a 20-second answer. | Parallelize independent agents (research ∥ tool calls); stream partial results to the user. |
You cannot improve what you don't measure — and agent teams fail in ways unit tests don't catch. A practical evaluation stack has four layers:
Collect 30–100 representative tasks with known-good answers. Run the full team against them on every change — prompts, models, and orchestration logic all regress silently otherwise. Track exact-match where possible and rubric scores where not.
Test each role in isolation with fixed inputs: does the planner produce valid, complete plans? Does the critic catch seeded errors? Deliberately inject known mistakes into drafts and measure the critic's catch rate — this is the single most informative metric in the system.
Use a strong model with a written rubric to grade outputs at scale, and spot-check a sample by hand to calibrate the judge. Crucially, the judge also audits the critic: sycophantic consensus (Section 10) is invisible unless someone grades the grader.
Log every message, tool call, and verdict as a structured trace. Review failed traces weekly: most systematic problems (role bleed, context poisoning, silent tool failures) are obvious in traces and invisible in aggregate metrics.
Before committing to the architecture, answer honestly. Check every statement that is true for your task:
To make this concrete, here is how the role pattern maps onto a real engineering use case — an assistant that documents Simulink / TargetLink motor-control software from model files, ARXML interface descriptions, and A2L calibration files:
| Agent | Job in this system |
|---|---|
| Ring Builder | Determines the task/ring context from the model, ARXML, and A2L files (the planner role, specialized for the domain). |
| Requirements Agent | Generates system and unit requirements (a doer). |
| Autodoc Agent | Writes subsystem behavior, inputs, outputs, and purpose (a doer). |
| Calibration Agent | Finds calibrations and signal dependencies (a researcher / tool operator). |
| Verifier Agent | Checks every statement against MDL/SLX/ARXML evidence (a critic, mechanical). |
| Critic Agent | Flags hallucinations, missing signals, and weak evidence (a critic, judgmental). |
| Supervisor | Decides when the documentation is acceptable or must be revised. |
Request: "Document the torque limiter subsystem." Follow the trace — note how every claim carries an evidence pointer, and how the verifier and critic act differently:
The video's idea is correct, but the real engineering challenge is not "adding more agents." It is agent governance — the rules that keep the team honest:
For enterprise engineering work, the winning execution order is:
That pipeline is far stronger than a loose "chat between agents" architecture — it keeps the creativity of LLMs while anchoring every result to verifiable ground truth.
Five questions covering the whole monograph. Pick an answer for each, then grade yourself: