A practical tour of LlamaIndex, AutoGen / AG2, Pydantic AI, Semantic Kernel, LangGraph and CrewAI — what they are, how they're shaped, the protocols binding them, and where the field is actually heading.
These tools all answer one question — how do I wrap a language model in enough structure to do useful, repeatable work? — but each answers it from a different starting point. LlamaIndex starts from your data. AutoGen / AG2 and CrewAI start from agents working together. Pydantic AI starts from type-safe contracts. Semantic Kernel starts from enterprise plumbing. LangGraph starts from an explicit state machine.
Worth saying up front, because it reframes everything below: the framework you pick matters less than the field once believed. The dominant trend is engineers assembling their own lightweight harness and treating the hard problems as cross-cutting concerns — evaluation (is it actually working?), observability (what did it do and why?), and memory (what carries forward?). As one practitioner put it bluntly, the gap between a good agent system and a bad one is almost never the framework — it's the eval pipeline, the observability setup, and the failure-recovery logic[32]. The frameworks below are increasingly judged by how cleanly they expose those seams. We return to this in §D–G.
01 · Data & Retrieval
LlamaIndex
The data framework that grew into agentic document processing.
LlamaIndex began as the connective tissue between an LLM and your corpus. Its identity is the retrieval pipeline: ingest, chunk, embed, index, and serve the right context at query time. The team has been explicit that they moved beyond being "just a RAG framework" toward best-in-class agentic document processing — OCR, structured extraction, and orchestration over documents[5].
Two pieces anchor the modern stack. LlamaParse handles hard real-world documents (tables, charts, mixed layouts) where naive OCR produces garbage. Workflows 1.0, released mid-2025, is a lightweight event-driven framework for agentic systems[2] — prebuilt agents for speed, or hand-authored workflows for full control[7]. The flagship pattern, Agentic Document Workflows (ADW), fuses parsing, retrieval, structured output and agent loops into end-to-end knowledge work, a step past both classic IDP and plain RAG[4].
Fig. 1 — Data flows left-to-right into the index; the agent loop pulls context back through the retriever.
// minimal shape
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
docs = SimpleDirectoryReader("./service_manuals").load_data()
index = VectorStoreIndex.from_documents(docs) # ingest → embed → index
engine = index.as_query_engine()
print(engine.query("What torque spec applies to the rotor bolt?"))
Practical notes
Reach for it when the hard part is the documents — messy PDFs, tables, A2L/ARXML-style artifacts, large heterogeneous corpora.
Hybrid stacks are common: use LlamaIndex for ingestion + retrieval, hand results to a different orchestrator for tool-heavy reasoning. You needn't adopt the whole thing.
Edge path exists: the open-source local route (filesystem tools, local embeddings) is the relevant one off-cloud — recent experiments show simple file exploration can beat RAG on small datasets[6].
Gotchas
LlamaParse and managed indexing sit behind a cloud/paid tier — costs scale with document volume, easy to under-budget.
Default chunking is rarely optimal for technical docs; tables and figures need tuned parsing or they fragment.
It is not an orchestration platform — for elaborate multi-agent control you'll bolt on another tool anyway.
Agents that solve problems by talking to each other — and a fork you need to understand.
AutoGen came out of Microsoft Research as a framework for conversable agents: compose specialized agents (plus an optional human proxy) into a group chat where they collaborate, debate, reflect, and execute code to reach a goal[11]. Group chat, sequential conversation, reflection, parallel execution — those patterns made it popular.
The thing to internalize is the governance split. The original authors (Chi Wang, Qingyun Wu) left Microsoft in late 2024 and forked the project as AG2, keeping the proven v0.2 architecture, the original PyPI packages (autogen, pyautogen, ag2) and independent governance under Apache 2.0[9][12]. Microsoft rewrote AutoGen as v0.4+ (async actor model), then in October 2025 folded it together with Semantic Kernel into the new Microsoft Agent Framework (MAF), leaving the classic line in maintenance[10].
Fig. 2 — A group chat orchestrates specialists; the project itself forked into AG2 and the MAF successor.
// AG2 / v0.2-style shape
from autogen import AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager
planner = AssistantAgent("planner", llm_config=cfg)
coder = AssistantAgent("coder", llm_config=cfg)
user = UserProxyAgent("user", human_input_mode="NEVER", code_execution_config={...})
chat = GroupChat(agents=[user, planner, coder], messages=[], max_round=12)
mgr = GroupChatManager(groupchat=chat, llm_config=cfg)
user.initiate_chat(mgr, message="Diagnose the 3000 RPM trip from this log...")
Practical notes
Reach for it when the natural decomposition is "several specialists arguing it out" — code-gen with a critic, plan-then-execute, debate/consensus.
Pick the branch deliberately. AG2 = stable 0.2 patterns, no Azure dependency. MAF = the Microsoft-blessed future with Entra ID, OpenTelemetry, Foundry hosting — natural if you live in Azure[10].
Gotchas
Three architectural shifts in under two years (0.2 → 0.4 → MAF). Production code carries a migration plan whether you want one or not[9].
Package-name confusion (autogen vs pyautogen vs Microsoft's repo) trips up dependency pinning — read which project you're installing.
Free-flowing conversation is harder to constrain in production; without round caps and tight tool boundaries, agents loop or wander.
"The FastAPI feeling" for agents — validation as a first-class citizen.
Built by the Pydantic team (Samuel Colvin and collaborators), Pydantic AI starts from a sharp observation: nearly every Python agent library already uses Pydantic for validation, yet none delivered the ergonomic, type-safe experience FastAPI brought to web work[20]. So the agent's output is a Pydantic model, its tools are validated functions, and its dependencies are typed and injected — pushing whole classes of error from runtime to write-time[17].
It's model-agnostic and ships the surrounding production machinery: Logfire for OpenTelemetry-native observability, Pydantic Evals for systematic testing, a graph layer, durable execution, plus MCP and A2A support[16][17]. If the schema fails to validate, the framework re-prompts the model in a self-correction loop[21]. It reached a stable 1.x in late 2025 and iterates roughly weekly[18].
Fig. 3 — Type-safe boundaries on every edge; the validation loop re-prompts until the output fits the schema.
// the whole idea
from pydantic import BaseModel
from pydantic_ai import Agent
classFault(BaseModel):
component: str
severity: int # constrain 0..10 and it's enforced
rationale: str
agent = Agent('anthropic:claude-opus-4-8', output_type=Fault)
result = agent.run_sync("Triage this CAN log: ...")
fault = result.output # a typed Fault, guaranteed-shaped
Practical notes
Reach for it when you want predictable, parseable output to hand straight to downstream code — exactly the constraint in safety-adjacent or pipeline-fed systems where a free-text blob is a liability.
Lightweight by design — closer to a typed harness than a heavy platform, so it fits the "build your own" trend rather than fighting it.
Gotchas
Weekly releases mean a fast-moving surface; pin versions and read changelogs even though the 1.x API is stable.
Single-agent first — multi-agent orchestration exists (graph layer) but is less battle-tested than AutoGen/LangGraph for complex topologies.
The retry-on-invalid loop can quietly multiply token cost on poorly-constrained schemas; watch Logfire traces.
Microsoft's model-agnostic SDK for putting LLM reasoning inside real applications.
Semantic Kernel (SK) launched in early 2023 as Microsoft's answer to "how do I integrate an LLM into a production app safely and repeatably?" Its mental model is three parts[25]: the Kernel is a dependency-injection container that orchestrates everything; Plugins (formerly "skills") are the typed tools you give an agent; and a Planner inspects the request and available plugins to build a step-by-step plan. It also carries early memory abstractions and runs with C#, Python and Java parity — its enterprise calling card[30].
Current reality: SK has been folded into Microsoft Agent Framework, which reached 1.0 in 2026 and unifies SK's enterprise foundations with AutoGen's orchestration[23][29]. Notably, MAF drops the explicit planner — modern reasoning models plan natively given the right tools — and makes MCP a first-class way to attach tool servers[26]. Microsoft has committed to supporting SK v1.x for at least a year past MAF's GA[28].
Fig. 4 — Kernel as the hub for plugins, planner and memory; the whole thing now flows into MAF.
// C# — the enterprise calling card
var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(deployment, endpoint, apiKey);
builder.Plugins.AddFromType<DiagnosticsPlugin>(); // [KernelFunction] toolsvar kernel = builder.Build();
var reply = await kernel.InvokePromptAsync(
"Summarize the fault codes and recommend next steps.");
Practical notes
Reach for it when you're on .NET / Azure or need C#/Java parity — the rare agent SDK that treats non-Python enterprise stacks as first-class.
Start new work on MAF, not raw SK. Microsoft is explicit MAF is the recommended path; SK knowledge transfers and a migration guide exists[28].
Gotchas
You're adopting a framework mid-transition — building fresh on legacy SK risks an avoidable migration to MAF later.
Historically verbose vs. Python-first frameworks; expect more boilerplate to reach the same behavior.
The classic Planner pattern is being retired — don't over-invest in bespoke planner logic that strong models now do natively[26].
From chaining LLM calls to an explicit, auditable state machine.
LangChain began as the framework for chaining LLM calls and grew into a sprawling platform — LangSmith for observability, LangServe for deployment, and LangGraph for stateful agent workflows — with one of the largest ecosystems in the space[37]. The LangChain team now explicitly steers people to LangGraph for agents and keeps LangChain itself for RAG and document Q&A[39].
LangGraph models a workflow as a directed (often cyclic) graph: nodes execute actions, edges define transitions, and state flows through and persists at checkpoints — giving explicit control over every decision point, retry and branch[31]. Production patterns that used to be community recipes — checkpointing, durable execution, human-in-the-loop approvals — are now first-class[14], with a stable Postgres checkpointer and first-class TypeScript support[38][35]. It's the default pick for regulated, auditable production where deterministic control matters[36].
Fig. 5 — Nodes and conditional edges over a persisted state; cycles, retries and human approvals are explicit.
// graph shape
from langgraph.graph import StateGraph, START, END
g = StateGraph(DiagState)
g.add_node("triage", triage_fn)
g.add_node("tools", tool_fn)
g.add_edge(START, "triage")
g.add_conditional_edges("triage", route, {"call": "tools", "done": END})
g.add_edge("tools", "triage") # cycle back
app = g.compile(checkpointer=PostgresSaver(...)) # durable state
Practical notes
Reach for it when you need auditability, deterministic control, retries, and human approval steps — regulated or safety-adjacent workflows where "what happened and why" must be reconstructable.
Incremental if you're already on LangChain — adding LangGraph reuses your RAG, tools and embeddings rather than maintaining two dependency trees[37].
Only major framework with first-class TypeScript, useful if your edge/UI layer is JS[35].
Gotchas
The graph paradigm has a real learning curve — it clicks for some immediately and confuses others for weeks[35].
Overkill for a simple linear workflow; don't reach for a state machine when a single typed agent call would do.
No native MCP/A2A protocol support yet — it leans on community integrations, where CrewAI ships them in-box[33].
The fastest path from idea to a working multi-agent demo.
CrewAI orchestrates teams of role-based agents. You give each agent a role, goal and backstory, assign it tasks, and assemble them into a crew that runs under a process type — sequential, hierarchical or consensual[31]. The team metaphor is intuitive and fast: most teams get a working multi-agent setup running in a few hours[35]. It's been the fastest-growing framework in the category, with ~45k GitHub stars, version 1.x, hundreds of millions of monthly workflow executions, and adoption across a large share of the Fortune 500[33][36].
Crucially for interoperability, CrewAI shipped native MCP and A2A support — making it a strong choice when your agents must reach external tool servers and talk to agents built on other frameworks[33][37].
Fig. 6 — Specialists with roles pass tasks down the crew; MCP and A2A connect outward.
// crew shape
from crewai import Agent, Task, Crew, Process
analyst = Agent(role="VHM Analyst", goal="diagnose the fault",
backstory="20 yrs in powertrain prognostics")
task = Task(description="Classify this bearing signature", agent=analyst)
crew = Crew(agents=[analyst], tasks=[task], process=Process.sequential)
result = crew.kickoff()
Practical notes
Reach for it when you want a working prototype this week and your problem maps cleanly onto "a team of specialists."
Best-in-class interoperability right now thanks to in-box MCP + A2A — relevant if you're federating across vendors or fleets[33].
Gotchas
The well-documented 6–12 month ceiling: teams move fast, then need non-linear flows or dynamic agent spawning, hit the opinionated design, and rewrite[39].
Every framework above now cites these two acronyms. They're not competing — they sit on perpendicular axes and are designed to work together[44].
MCP (Model Context Protocol) is an open standard from Anthropic — think "USB for AI." It standardizes the vertical link: how a single agent connects to external tools and data (APIs, databases, file systems) through one consistent interface, over JSON-RPC, so you stop writing bespoke glue for every tool and every model[40][45].
A2A (Agent-to-Agent) is an open standard introduced by Google (April 2025, 50+ partners). It standardizes the horizontal link: how agents discover and collaborate with each other across vendors and frameworks. Agents advertise capabilities via an Agent Card (a JSON resume), and it's intentionally stateful — a Task object moves through states like submitted → working → input-required so both sides share one view of long-running work[41][42][47].
Fig. 7 — Perpendicular and complementary: MCP wires an agent to its tools; A2A wires agents to each other.
Why this matters for fleets
If you're designing agents that must outlive any single framework choice, build to the protocols, not the SDK. An MCP server you write once (say, a CAN/UDS query tool) is reusable by any MCP-aware framework; an A2A-exposed edge agent can be coordinated by a cloud orchestrator regardless of what either is built on. That's the cleanest hedge against the migration churn the framework sections keep flagging.
The six frameworks reduced to the axes that actually drive a choice. "Status" reflects the field as of early-to-mid 2026; treat star counts and versions as directional (see §H).
Framework
Starts from
Sweet spot
Languages
Status / caution
LlamaIndex
Your data
RAG, document parsing, agentic doc workflows
Python, TS
Active; cloud tiers cost money; local path for edge
AutoGen / AG2
Agents conversing
Multi-agent collaboration, code-gen, debate
Python
Forked: AG2 vs MAF — pick deliberately
Pydantic AI
Type-safe contracts
Structured outputs, lightweight typed harness
Python
Stable 1.x; weekly releases; Logfire/Evals
Semantic Kernel
App plumbing
Enterprise / .NET / Azure integration
C#, Python, Java
Superseded by MAF 1.0; SK v1.x supported ~1yr past GA
A rough triage, not a verdict. The honest first question is whether you need a framework at all — for a single typed call, a thin harness plus the protocols often wins.
Fig. 8 — Walk top-to-bottom; the first "yes" is your strong default, not a rule.↑ top§ D — WHERE THE FIELD IS GOING
Past the framework wars
Notice what every project converged toward in 2025–26: native MCP for tools, native observability, built-in evaluation, explicit memory — while clever orchestration DSLs receded (MAF even dropped the planner). That convergence is the signal. The differentiating problems moved out of the framework into three cross-cutting concerns.
Fig. 9 — The framework is interchangeable; the three concerns below it are where the engineering effort now lives.↑ top§ E — REFERENCE ARCHITECTURE
Mapping it onto a three-tier system
For a constrained compute hierarchy (a safety-critical controller, a high-compute edge node, a cloud backend) the lens shifts usefully. The rule of thumb: no framework belongs on the safety-critical tier — that stays deterministic. The edge earns a thin typed harness; the heavy orchestration lives in the cloud where compute and observability are cheap.
Fig. 10 — Triage and enrich at the edge before the bandwidth-limited uplink; orchestrate and learn in the cloud.
Unifying rule
The same lesson MAF stumbled into applies at every tier: give good tools and tight schemas first; build bespoke orchestration last. Pydantic AI's "never emit an unparseable blob" discipline is exactly what you want feeding a Tier-1 boundary; LangGraph's auditable state machine is what you want when a cloud decision must be reconstructable months later.
This is where §D's three concerns become a control loop. A new or updated "skill" (a tool, a prompt, a sub-agent) should never ship to a fleet on author confidence alone. Evaluation and observability are the gates; telemetry from the field closes the loop and produces the next batch of training/eval data.
Fig. 11 — Two gates (automated eval, then human sign-off after a shadow run), then staged OTA; field telemetry regenerates eval data.
Eval gate blocks anything that regresses a golden set — cheap, automated, runs on every candidate.
Shadow/canary exercises the skill on live traffic without acting, so observability catches surprises before they reach Tier 1.
Human sign-off produces the audit record — the artifact a safety case needs, and exactly what LangGraph/MAF HITL checkpoints are built to capture.
Telemetry → weak labels → golden set is the flywheel; the adaptive-acquisition campaign decides what new data to pull next.
Every framework now ships some eval hook (Pydantic Evals, LangSmith datasets, CrewAI's observability), but the substance is yours to build. A workable shape:
1 · A golden set with teeth
Start small and adversarial: 30–100 cases that each pin one behavior, with a known-good answer or an assertable property. For diagnostic work that's often a property, not a string — "severity is within ±1 of ground truth," "names the correct subsystem," "never fabricates a DTC." Grow it from field failures (§F's flywheel), not from cases the system already passes.
2 · Scoring that isn't just string-match
Layer three kinds of check: deterministic assertions (schema valid, value in range — Pydantic does this for free), reference metrics where a gold answer exists, and an LLM-as-judge for open-ended quality with a tight rubric. Keep judge prompts versioned; a drifting judge silently moves your bar.
3 · Regression discipline
Run the whole set on every skill candidate and store scores per version. The gate is relative, not absolute: block anything that drops a previously-passing case, even if aggregate score rises. Aggregate improvements routinely hide specific regressions.
4 · Cost and latency as first-class metrics
Score tokens and wall-clock alongside quality — a skill that's 2% better and 3× slower may be a net loss at the edge. Observability traces (OTel via Logfire/LangSmith) give you these for free if wired early.
# sketch: property-based eval over a golden setfor case in golden_set:
out = agent.run_sync(case.input).output # typed → already schema-valid
record(case.id, version,
in_range = abs(out.severity - case.truth.severity) <= 1,
subsystem_ok = out.component == case.truth.component,
no_fabricated_dtc = out.dtc in KNOWN_DTCS,
tokens = trace.tokens, latency_ms = trace.ms)
gate_pass = no_regressions(version) and aggregate(version) >= THRESHOLD
Star counts and versions move weekly and vary by source; these are representative figures verified June 2026, not authoritative. Always check the primary docs in §R before pinning anything.