AI Agent Architecture: Components, Patterns and How to Choose
AI agent architecture is a decision about where control and state live: who chooses the next step, what is written down between steps, and what checks it. Pick the simplest arrangement whose checks you can actually run, and add agents only to solve a problem you can name.
The example in this article is a composite scenario built from patterns common to insurance claims operations. It is not a single real organisation. Its counts come from two short scripts published with the article, and follow from the designs as written.
A claims team at a home insurer wanted an agent to handle straightforward contents claims: a stolen bike, a water-damaged laptop. The proposal arrived as an architecture diagram with six boxes. A supervisor agent would receive each claim and delegate to five specialists: an intake agent, a policy agent, a fraud agent, a payout agent and a letter agent. Each specialist mirrored a job on the claims floor, and the demo, run on a dozen hand-picked claims, was convincing.
The architecture review asked one question the diagram couldn't answer: what does each agent know when it decides? The fraud agent saw a two-paragraph summary the supervisor had written, not the claim. The payout agent saw a coverage finding and a fraud score, not the policy wording or the receipts. The letter agent saw a decision summary. Seven of the ten arrows on the diagram carried a summary written by a model rather than the evidence itself, and nobody had tested what those summaries left out.
The rebuild kept the same six tools and replaced five of the six agents with one agent inside a runtime that owns the claim's state. This article explains why, sets out the patterns you can choose from, and gives a method for choosing.
What AI agent architecture covers
"Agent architecture" is used for three different things, and most arguments about it are two people talking about different layers.
The components inside one agent. A model, instructions, the context assembled for each decision, tools and credentials, autonomy per action, state, stop conditions and escalation, traces, and tests. AI Agents: What They Are and Why They Keep Failing maps all ten, each with the failure it prevents. This article doesn't repeat that map; it assumes it.
The topology. How many agents or model steps there are, how they are connected, and who decides what happens next: code, one model, or several models negotiating. This is what most "AI agent architecture diagrams" show, and it is where the patterns in this article live.
Runtime control. Where the system's state is held, what moves it from one state to the next, and what is checked before it does. This layer rarely appears on the diagram. It decides whether the system can be tested.
The diagram above shows the reference architecture for one production agent, and the patterns built from it. The single most important line on it is the one between the runtime and the model. The model proposes; the runtime decides whether the proposal is allowed, records it, and moves state. When that line is missing, the model's own account of what it has done becomes the system's only record, and that account is what fails first.
The test: take your current architecture diagram and mark which box holds the task's state. If the answer is "the model's context" or "each agent keeps its own", nothing outside a model can say what has been checked.
Why the org-chart design is attractive
The six-agent proposal is a common first design, and it is attractive for honest reasons.
It mirrors the work. A claims floor really does have intake, coverage, fraud, settlement and correspondence roles, so a diagram with one agent per role is easy for the business to read and easy to sign off. It promises specialisation: each agent gets a short, focused prompt instead of one long one. It makes for good demos, because each agent can be shown doing its job in isolation. And it is cheap to build: some agent frameworks are organised around defining agents by role and having them talk to each other.
The promise is also partly real. Anthropic reported that its multi-agent research system, a lead agent delegating to parallel subagents, outperformed a single agent by 90.2% on its internal research evaluation. The same write-up is careful about where that holds: tasks with "heavy parallelization, information that exceeds single context windows", and "breadth-first queries that involve pursuing multiple independent directions simultaneously" (Anthropic, 2025).
A contents claim is none of those things. It is a short, sequential task where every step depends on the previous one and all of the evidence fits comfortably in one context window.
The test: for the work you are designing for, ask whether its parts could be done by people who never talk to each other. If not, splitting it across agents means building the conversation they would have needed.
What the org chart misses: every boundary is a hand-off
An agent boundary is a hand-off, and a hand-off between models has three properties that a hand-off between people on a claims floor usually doesn't.
The receiver sees only what the sender wrote down. A claims handler who is unsure can walk over and ask. A worker agent gets a task brief and its own context, nothing more. Cognition's engineers state the first principle of agent design as "share context, and share full agent traces, not just individual messages", and the second as "actions carry implicit decisions, and conflicting decisions carry bad results" (Yan, 2025). A summary is a lossy copy of the evidence, written by a model that doesn't know which detail the next agent will need.
Failures cluster at the boundaries. One systematic study annotated more than 1,600 traces from seven multi-agent frameworks and found 14 failure modes in three groups: system design issues, inter-agent misalignment and task verification (Cemri et al., 2025). Two of the three groups are about the arrangement, not the model: agents misunderstanding each other, and nobody verifying the combined result.
It costs more to run. Anthropic's figures: agents typically use about 4× more tokens than chat interactions, and multi-agent systems about 15× more. It found token usage by itself explained 80% of the performance variance in its research evaluation (Anthropic, 2025). Spending more tokens buys breadth when the task is broad. On a narrow, sequential task it buys duplicated reading.
The same write-up names the operational cost too: agents are stateful and errors compound, and minor changes can cascade into large behavioural changes. Every additional agent is another prompt, another context and another place for those changes to start.
The test: count the arrows between agents on your diagram, and for each one write what crosses it. Every arrow that carries a model-written summary instead of the source record is a place where evidence can be lost without anyone noticing.
The patterns, from least to most autonomy
These are the patterns worth knowing, ordered by how much of the control flow the model owns. Each step down the table adds capability and test surface.
Pattern | Who decides the next step | Use when | Signature failure | How to test it |
Single agent with tools | One model, inside runtime rules | The default for most tasks | Claims checks it never ran | Force a tool failure; the run must hand over |
Fixed workflow (chain, router, parallel) | Code | The steps are known in advance | A wrong value passed on as fact | A check on every hand-off; end-to-end sample |
Planner–executor | A model writes the plan; code or a model executes it | The plan should be inspectable before acting | Executing the rest of the plan after a step failed | Fail step N; the run must replan or stop |
Generator–critic | A model loop | Output quality can be checked against criteria and evidence | The critic passes plausible work it can't verify | Seed known-bad drafts; measure the catch rate |
Supervisor–worker (orchestrator) | A lead model delegates to workers | Independent subtasks too large for one context | A worker's wrong result accepted; a subtask never created | Inject a faulty worker; check coverage against a required list |
Peer multi-agent | Several models negotiate | Research settings; rarely production | Agents talk past each other; nobody verifies the whole | Hard to test well, which is the warning |
Fixed workflows are covered pattern by pattern in Agentic Workflows: The Five Patterns and Where Each One Breaks, including Anthropic's orchestrator-workers pattern, which is the supervisor–worker row seen from the workflow side. Whether you need an agent at all, or a workflow will do, is the question in Agent, Workflow, Automation or RPA?.
Three patterns deserve a note, because their failures are easy to miss.
Planner–executor makes the plan visible, which is its value: a reviewer or a rule can reject a plan before anything acts. Its failure is staleness. The plan was written before step two returned, and an executor that keeps going after step three fails is executing a plan for a world that no longer exists. The hands-on version is Build a Planning Agent.
Generator–critic works when the critic can see what the generator should have used. A critic that reads only the draft checks tone and consistency; one that reads the draft and the evidence checks truth. Research on self-correction found language models struggle to correct their own reasoning without external feedback (Huang et al., 2024), so give the critic the evidence record, not just the output.
Supervisor–worker is the right answer when the work genuinely splits: reviewing two hundred contracts for one clause, researching ten independent questions. Its failures are acceptance (the supervisor integrates a worker's wrong result) and absence (a subtask that should have existed was never created). Test both directly. Multi-Agent Workflows builds one.
The test: name the pattern your system actually uses, which may not be the one in the design doc, and run the test in its row this week.
The claims agent, rebuilt
The review compared the two designs as data rather than as diagrams. Each design was written down as its agents, the arrows between them and what crosses each arrow, plus the tools. The published script counts what follows from each design.
== design A: supervisor + 5 workers (the proposal)
agents 6
hand-offs between agents 10
of which pass a summary, not evidence 7
tools 6
model calls, straightforward claim (min) 11
== design B: one agent + runtime state machine + letter critic (the rebuild)
agents 2
hand-offs between agents 2
tools 6
model calls, straightforward claim (min) 6
(Excerpt of output.txt. The model-call figures are minimums under the assumptions stated in architectures.py: one call per worker task, and a supervisor call to plan and to integrate each result.)
Both designs use the same six tools, so the system's reach is identical. What changed is the number of places where evidence passes through a model-written summary: seven in the proposal, none in the rebuild. The rebuild's only agent-to-agent hand-off is the letter critic, and it receives the draft and the evidence record.
Two other changes came out of the review, and both are discovery moves rather than architecture choices. The fraud agent became a deterministic rule step, because the claims team's fraud referral criteria were already written down as rules; nobody had asked to see them before the design was drawn. And the payout limit became a transition in code rather than an instruction in a prompt, because the adjusters' authority limit was a fixed number in their delegation policy.
The rebuild's runtime is a state machine. The model chooses at five decision points; the runtime refuses any move that isn't on the list, and every allowed move has a written guard.
ALLOWED = { # state -> {next state: guard that must hold}
"received": {"triaged": "claim and policy records loaded"},
"triaged": {"evidence": "route is simple or complex",
"handed_over": "route is refer"},
"evidence": {"assessed": "every required document present",
"handed_over": "a required document is missing after one request"},
"assessed": {"drafted": "fraud rules returned clear",
"handed_over": "fraud rules returned refer"},
# ...
== runtime state machine (design B)
states 9
decision states (model chooses) 5
possible transitions between states 72
allowed, each with a written guard 13
refused by the runtime 59
example refusal assessed -> closed
(The Python is an excerpt of claim_runtime.py; the # ... line elides the remaining four states. The output is the second half of output.txt.)
That last line is the point of the design. In the proposal, nothing stopped the supervisor from closing a claim whose fraud check never ran, because "has the fraud agent answered?" lived only in the supervisor's context. In the rebuild, assessed -> closed is not a move the runtime will make. Thirteen allowed transitions with written guards is also a test plan: one test per transition, plus one that confirms an illegal move is refused. Agent Memory and State covers how to keep state in the runtime rather than in the model.
The test: write your system's states and allowed transitions as a table. If you can't, the architecture has no runtime layer, and the model is the only thing deciding when a task is finished.
How to choose: four questions, in order
Ask these for each task the system handles, not once for the whole system. Stop at the first answer that settles it.
Is the path known in advance? If yes, use a fixed workflow with a model step where the input is messy. Most of a claims process is known in advance. Evidence to bring: the current procedure document or the process map.
Does the task fit in one context, with one owner of the outcome? If yes, use a single agent with tools inside a runtime that owns state. This is the right answer far more often than the frameworks' examples suggest. Evidence: the size of the inputs for a large case, in tokens, against the model's window.
Does it split into parts that are genuinely independent? Independent means no part needs another's result or assumptions. If yes, a supervisor–worker design may earn its cost; make the supervisor check coverage against a list and pass full records, not summaries, to workers. Evidence: a dependency map of the subtasks.
Can each output be checked against evidence? If yes, add a critic where a check in code isn't possible, and give it the evidence. If no, the output needs a person, at the autonomy level set in Where Human Approval Belongs.
The claims team's answers were: partly (triage and fraud rules were known), yes (a contents claim fits easily in one context), no (every step depends on the previous one) and yes for the letter. That sequence produced design B.
Tradeoffs and exceptions
Option | Gains | Costs | Best suited for | Warning signs |
Single agent + runtime state machine | Fewest hand-offs; testable transitions; one context | One long prompt to maintain; context limits on very large tasks | Sequential tasks with one owner | State kept only in the model's context |
Fixed workflow | Cheapest to run and test | Can't handle paths it didn't anticipate | Known procedures with messy inputs | Exceptions piling up in a manual queue |
Planner–executor | Plans can be reviewed before acting | Plans go stale; extra calls | Multi-step tasks where reviewing the plan is cheap | Executor never replans |
Generator–critic | Catches defects before release | Doubles calls; a blind critic adds false confidence | Outputs with clear criteria and an evidence record | First-pass approval near 100% |
Supervisor–worker | Breadth; parallel speed; work beyond one context | Several times the tokens; coordination failures | Many independent subtasks | Workers receive summaries, not records |
Peer multi-agent | Exploration in research settings | Hard to test, debug or attribute | Experiments | Proposed because it mirrors the org chart |
The main exception to "start with one agent" is size. When a single case's evidence genuinely exceeds what one context can hold, splitting the reading across workers is the honest fix, and Anthropic's research system is the documented example. The second exception is separation of duties: if policy requires that the party who proposes a payment cannot approve it, two separate identities may be a control requirement, not a design preference. In both cases, keep the hand-offs few and pass records rather than summaries.
Architecture decisions are also more expensive to reverse than prompt decisions. Prompts change weekly; the state model, the tool gateway and the number of agents tend to set for the life of the system. Spend the review time before the build, not after the first incident.
The architecture review: a method
The review takes half a day with the system's owner, someone who does the work today, and whoever will test it. It produces an architecture decision record the team can test against.
Write each candidate design as data. Agents, arrows, what crosses each arrow, tools. Done when every arrow names what it carries, and "context" is not accepted as an answer.
Ask the four questions per task type, with the evidence each one names. This is where the discovery happens: the claims team found its fraud rules and the adjusters' authority limit in this step. Done when each task type has an answer and its evidence.
Write the runtime's states and allowed transitions, each with a guard. Done when a reviewer can point at any state and say what must be true to leave it.
Count the hand-offs and price them. Arrows carrying summaries, model calls per typical case, and which tests each arrow needs. Done when the chosen design's hand-offs each have a named test.
Record the decision and what would change it. For example, "move to supervisor–worker if a single case's evidence exceeds the context window". Done when the record names its own trigger for review.
The record feeds directly into the test plan described in How to Test an AI Agent, and each transition becomes a step the trace should record, using the fields in The AI Agent Observability Checklist.
How architecture goes wrong
Agents that mirror job titles. The org chart is a map of how people coordinate, and models coordinate differently. Design from the work's dependencies, not its roles.
State in the model's context. The system's only record of what was checked is a model's recollection. Move it to the runtime.
Summaries across boundaries. Each hand-off passes a lossy copy. Pass the record, or don't split.
A critic without evidence. It approves fluent work. Give it the source record, and track how often it passes on the first try.
A pattern chosen for the demo. Multi-agent demos well because each agent can be shown alone. Production failures happen between them. The Agent Failure Mode Catalogue lists unsafe hand-offs between agents as one of eighteen modes to test.
Where to go next in this path
This page is the hub for the architecture path of the Agent Engineering curriculum.
The parts inside one agent: AI Agents: What They Are and Why They Keep Failing
Whether you need an agent: Agent, Workflow, Automation or RPA?
Workflow patterns and their hand-offs: Agentic Workflows: The Five Patterns and Where Each One Breaks
Hands-on builds: Tool Use and Agents, Build a Planning Agent, Multi-Agent Workflows
Designing from the checks backwards: Verification-First Agent Design
The arrow between the model and its tools: Designing Tool Contracts and Least Privilege for AI Agents
What to do next
Take the agent system you are building or reviewing and do step 1 of the method this week: write it down as agents, arrows and what crosses each arrow. Count the arrows that carry a model-written summary. If the count is above zero, ask of each one whether the receiving agent could be given the record instead, or whether the two agents should be one.
Final takeaway
An AI agent architecture is good when something other than a model can say what has been checked and what is allowed next. Start with one agent inside a runtime that owns state and refuses moves it doesn't allow. Add a pattern only to solve a problem you can name, pass records rather than summaries across every boundary you add, and test each boundary on purpose.
Sources
Anthropic, How we built our multi-agent research system, 13 June 2025. 90.2% improvement over single-agent on the internal research evaluation; about 4× (agents) and 15× (multi-agent) the tokens of chat; token usage explains 80% of variance; suited and unsuited tasks; stateful, compounding errors.
Yan, W. (Cognition), Don't Build Multi-Agents, 12 June 2025. Share full context and traces; actions carry implicit decisions.
Cemri, M., Pan, M. Z., Yang, S. et al., Why Do Multi-Agent LLM Systems Fail?, arXiv:2503.13657, 2025. 1,600+ annotated traces across 7 frameworks; 14 failure modes in 3 categories.
Anthropic (Schluntz, E. and Zhang, B.), Building effective agents, 19 December 2024. Workflows vs agents; orchestrator-workers.
Huang, J., Chen, X., Mishra, S. et al., Large Language Models Cannot Self-Correct Reasoning Yet, ICLR 2024. Self-correction without external feedback.
Frequently Asked Questions
What is AI agent architecture?
AI agent architecture is how an agent system is put together: the components inside each agent (model, instructions, context, tools, state, stop conditions, traces), how many agents or steps there are and who decides what happens next, and where state is held and checked between steps.
Should I use a single agent or multiple agents?
Start with a single agent with tools inside a runtime that owns state. Move to multiple agents only for a named reason, usually work that splits into independent parts too large for one context window. Every boundary between agents is a hand-off you have to test, and multi-agent systems cost more to run.
What are the main AI agent architecture patterns?
In rough order of autonomy: a single agent with tools, fixed workflows (prompt chaining, routing, parallelization), planner–executor, generator–critic, supervisor–worker (orchestrator) and peer multi-agent systems. Each one adds hand-offs and a characteristic failure mode, so each needs its own test.
Part of the Agent Engineering curriculum: eight paths from prompts and context to testing, evaluation and governance.
