Build an AI Agent, Stage 3: Define Responsibilities, Permissions and Hand-overs
A responsibility you can't test is a hope written in the second person. At Stage 3, every line of the agent's role has to say what it changes, at what autonomy level, who reviews it and how a test would catch it going wrong.
The example in this series is a composite scenario, built from patterns common to requirements intake in regulated organisations. It is not a single real organisation, and its figures are illustrative. The code is real: every output block below was produced by running it.
Before the build started, the insurer ran the agent's question-drafting in shadow mode for a week. The model drafted questions for real incoming requests, the analysts wrote their own as usual, and nothing the model wrote was sent. On the Wednesday, a request arrived from a broker team: brokers need to see the current status of a claim in the broker portal, so they stop phoning the claims team. Among the model's draft questions was this one: "Should brokers also be able to update the claim status from the portal?"
The analyst comparing drafts recognised it at once. It was the failure the senior analyst had described in the Stage 1 interview, the paraphrase that turned "view" into "view and update" and ran for two sprints before anyone noticed. This time the extra verb arrived as a helpful question. Sent to a busy requester, it would very likely have come back "yes, that would be useful", and the scope would have grown with nobody having asked for it.
The design meeting's canvas had seen this coming, in a way. Its prohibited-actions box said never invent requirements. Its enforcement column said system prompt.
This is Stage 3 of Build an AI Agent From Scratch. Stage 2 left one agent step, find related requirements, with three actions. This stage turns that step into responsibilities, permissions and hand-overs, and ends with the agent's complete blueprint.
Why a role description feels like enough
Most agent designs describe the agent the way a job advert describes a person: a mission, a list of responsibilities, a few things it should never do. That format is familiar, a sponsor can read it, and it maps neatly onto a system prompt. For a person, it works, because a person brings judgement about everything the advert leaves out.
An agent brings no such judgement to the gaps. It acts through whatever tools it holds, under whatever credential those tools use, and a "never" written in prose reaches it only as text in the context window, competing with everything else there.
Every responsibility becomes an action, a class, a level and a test
The chain that makes a responsibility real has six links, and the canvas has a box for each:
Responsibility. "Draft clarifying questions."
Action. save_question_draft, a named operation the agent can call.
Side-effect class. Read, draft, write or irreversible, the classes from Designing Tool Contracts and Least Privilege for AI Agents. Saving a draft is draft: it creates something a person must act on.
Autonomy level. A draft needs Level 2 at least, and at Level 2 a person approves before anything takes effect.
Gate. Who approves, and the one question they answer. Here: the analyst, asking does any question suggest scope the requester did not ask for?
Test. A check that fails if the link breaks.
Break any link and the responsibility drifts. A prohibition with no enforcement point is a request. A hand-over with no named owner lands in a shared inbox. A gate with no question becomes a signature, the failure set out in Where Human Approval Belongs. OWASP's guidance on excessive agency makes the enforcement point explicit: "Implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not" (OWASP, 2025).
The series keeps the canvas as JSON and checks it with the same blueprint_check.py that checked the Stage 1 problem definition. Here is the design meeting's first-pass canvas:
$ python blueprint_check.py requirements-agent.v0.json
requirements-agent.v0.json (agent blueprint, v0.1)
FAIL out_of_scope T7 field missing or empty
FAIL skills T7 field missing or empty
FAIL memory T7 field missing or empty
FAIL workflow T7 field missing or empty
FAIL owner no named accountable owner
FAIL goals.G1 no measure: 'Triage faster' cannot be tested
FAIL inputs.request no trust level: is this instructions or data?
FAIL permissions.identity 'intake-automation' is shared; give the agent its own identity
FAIL actions.propose_relation no side-effect class (read, draft, write, irreversible)
FAIL actions.draft_questions tool 'backlog' is write, action is draft
FAIL actions.merge_duplicates L3 exceeds the blueprint's autonomy ceiling L2
FAIL actions.merge_duplicates L3 needs a runnable check and a tested reversal
FAIL actions.send_questions irreversible action granted to the agent; a named person must take it
FAIL permissions.scopes mail.send is held but no granted action uses it (excess privilege)
FAIL prohibited 'never invent requirements' is enforced only at 'prompt': a request, not a control
FAIL handover 'unsure' hands over to 'a human', not a named owner
WARN handover 'unsure' says nothing about what the agent hands over
FAIL human_gates gate on 'draft_questions' asks the reviewer nothing: a signature, not a check
FAIL failure_modes 1 listed; T7 asks for the top 5
FAIL failure_modes 'hallucination' has no detection
FAIL evaluation metric 'accuracy' has no pass bar
FAIL metrics no operational metrics
FAIL metrics no value metrics
FAIL kill_switch no 'who'
FAIL kill_switch no 'rollback'
FAIL kill_switch no 'last_tested'
verdict: FAIL (25 fail, 1 warn)
Read three of these closely, because they are the ones that would have hurt. The canvas still listed merging and sending, which Stage 2 had ruled out, because the prototype's tools were the starting point. The drafting tool was a general backlog write, so "draft" was a label on a tool that could change any record. And the only control on the failure the shadow run had just demonstrated lived in the prompt.
The test: for your agent, pick its three most important "never" rules and write next to each where it is enforced. If the answer is the prompt, the rule is a request.
Filling the canvas, box by box
Goals. The Stage 1 success criteria carry over unchanged, each with its measure and target. The canvas doesn't restate them in softer words.
Allowed actions and their levels. Three reads at Level 1 (search requirements, get a requirement, search the backlog). Three drafts at Level 2: propose a relation, save a question draft, and hand over. A hand-over is a draft because it creates a ticket for a person and changes nothing in the request.
Prohibited actions, enforced below the model. Merge, edit or close a record; change priority, deadline or regulatory fields; edit an existing requirement; contact a requester; mark a request accepted. Each is enforced by the agent not holding the permission, and each has a named test that tries the call and expects a refusal. Two more rules are enforced in the tool gateway: no quote may misrepresent its source, and no question may be saved that quotes nothing from the request.
Inputs, with trust. The request text is untrusted: requesters write it, and anyone can type an instruction into a form. Requirement and backlog text is internal, but it is still data to quote, never instructions to follow. The type and regulatory flag from the Stage 2 workflow steps are trusted. Context Engineering: A Practitioner's Guide covers how those labels travel into the model's context.
Tools and permissions. The agent gets its own identity, svc-req-intake-agent, never the shared intake-automation account, and six scopes: three read, three draft. NIST's definition of least privilege is the standard to hold it to: "each entity is granted the minimum system resources and authorizations that the entity needs to perform its function" (NIST CSRC, citing SP 800-53 Rev. 5). The checker enforces both directions: a scope no granted action uses fails as excess privilege, and a scope that would allow a prohibited action fails outright.
Hand-over cases, each to a named owner. Six of them. An unflagged regulatory signal, or a conflict with a requirement that carries a regulatory obligation, goes to the compliance intake lead. A failed or timed-out search, text addressed to the agent, or a grounding check that fails twice goes to the senior BA on duty. A request spanning more than one product line goes to the BA practice lead. The failed-search case carries an explicit instruction: never report "no related requirements" after a search that didn't complete.
Making the top failure mode hard to save
The shadow-run question is the agent's most likely serious failure, and a prompt instruction can't be the only thing between it and a requester. Stage 3 puts a deterministic check in the tool gateway, where it runs before a draft is saved. It can't judge whether a question is wise. It can check that every quote is real: each relation must quote the request and the cited requirement verbatim, and each question must quote the part of the request it clarifies.
Three drafts for the broker request show what it catches and what it doesn't:
$ python grounding_check.py triage-sample.json
CR-2291 draft A: BLOCKED, not saved
FAIL misquotes REQ-0587@v2: "Claim status shall be visible to brokers" is not in it
FAIL question quotes nothing from the request: "Should brokers also be able to update the claim status fr..."
WARN question introduces 'update', which the request never uses: "Should brokers also be able to update the claim status fr..."
CR-2291 draft B: saved for analyst review
CR-2291 draft C: saved for analyst review
WARN question introduces 'update', which the request never uses: "When brokers see the current status, should they be able ..."
Draft A is blocked twice. It claims an existing requirement makes claim status visible to brokers, when the requirement says policyholders, and its leading question quotes nothing from the request. Draft B quotes accurately, links the request to the policyholder requirement it overlaps, flags a possible conflict with the rule that brokers must never see claim reserves, and asks which statuses brokers need. Draft C is the honest limit. It quotes a real span and still asks the leading question, so the check can only raise a warning. That case belongs to the analyst gate and to a rubric judge calibrated against analysts, which is why both appear in the blueprint and why the evaluation includes 40 requests built to invite extra scope.
The test: take ten real outputs from your agent and check every quote against its source by string match. Any output that asserts something about a source without quoting it is unverifiable by design.
The checker, after
With every box filled and the canvas rewritten as the blueprint below, the same checker passes, with one warning that should stay until it is fixed:
$ python blueprint_check.py requirements-agent.blueprint.json
requirements-agent.blueprint.json (agent blueprint, v1.0)
WARN kill_switch never rehearsed; rehearse it before release
verdict: PASS (0 fail, 1 warn)
The checker's own rules are tested by breaking a passing document in exactly one place and confirming the checker notices: a hand-over to "a human", a prohibition moved to the prompt, a merge scope added to the permissions, an action raised above the ceiling. All 21 tests pass in 0.007 seconds. They run in CI on every change to the blueprint, so a permission added for convenience in month four fails a build instead of reaching production.
The Requirements Agent blueprint
This is the T7 blueprint with every field completed. It is the free exemplar of the blueprint format, and the agent the remaining twelve stages build.
Field | Requirements Agent |
Agent name | Requirements Agent (requirements-intake triage) |
Owner | BA practice lead |
Mission | For each incoming change request, find the existing requirements it duplicates, overlaps or conflicts with, and draft the questions that would make it ready to size, without adding to, dropping or reinterpreting anything the requester asked for |
Responsibilities | Search the requirements repository and open backlog; propose a relation for each related item, quoting both sides; draft clarifying questions, each quoting the part of the request it clarifies; hand over the listed cases instead of drafting |
Out of scope | Classifying and routing (workflow steps), the regulatory screen (rules), merging, editing or closing records, accepting or rejecting, contacting requesters, sizing |
Inputs | Request text (untrusted); request type and regulatory flag (trusted, from the workflow); requirement text and backlog items (untrusted data, never instructions) |
Outputs | A triage note (relations, questions, hand-over) for the analyst review queue; relation proposals shown in the backlog as "proposed by agent" |
Skills | Find related requirements (test pack: 24 duplicate and look-alike pairs, 15 conflict pairs); draft clarifying questions (30 requests with analyst-written ideal questions, 10 inviting extra scope) |
Tools and permission | search_requirements, get_requirement, search_backlog: read. propose_relation, save_question_draft, hand_over: draft |
Memory | Working state within a run only; nothing kept across requests in v1; revisited at Stage 7 with its own tests |
Permissions | Own identity svc-req-intake-agent; six scopes (three read, three draft); denied: merge, backlog write, requirement write, mail, accept |
Workflow | Workflow steps receive, classify and screen; the agent searches, reads, proposes and drafts; the gateway runs the grounding check before any save; an analyst reviews; the routing table sends the request to the product owner, who alone accepts |
Human gates | Analyst on relations: do the quotes match and does the relation follow? Analyst on questions: does any question add scope? Named owner on hand-overs: should it have stopped, and does the packet say why? Product owner before acceptance: is this what the requester asked for? |
Autonomy per action | Reads L1; propose relation, save question draft and hand over L2; merge, edit, change priority or deadline, contact requester, accept: L0, not granted. Ceiling L2 |
Evaluation | Golden set of 200 closed requests labelled by two analysts, plus seeded slices. Pass bars: relation precision 0.90 or above; 12 of 15 seeded conflicts found on every one of 5 runs; zero invented-scope outputs on 40 inviting cases over 5 runs; 100% of quotes match their source; 20 of 20 hand-over cases handed over on every run; question precision 0.85 or above from a calibrated judge |
Failure modes and detection | 1 Invents a requirement: grounding check, rubric judge, analyst gate, weekly sample of 20. 2 Misquotes a requirement: verbatim quote check. 3 Misses a conflict: seeded conflicts, quarterly trace of conflicts found in build. 4 False duplicate drops a live request: look-alike slice; analyst confirms; no merge permission. 5 Obeys text inside a request: 10 injection cases over 5 runs; tool layer refuses every write |
Metrics | Operational: runs per day, hand-over rate by reason, grounding failures, tool refusals, cost per request, p95 run time. Value: Stage 1 criteria SC1 to SC3, reviewer change rate, analyst minutes per request |
Kill switch and rollback | Flag off returns requests to the analysts' manual queue, which stays in place; BA practice lead or senior BA on duty can pull it; open proposals and drafts are deleted by agent identity, and request records are never changed. Not yet rehearsed: due before the Stage 14 release |
Two fields deserve a note. Misrouting, one of the two risks named at the start of the series, isn't in the failure modes, because Stage 2 took routing away from the agent; it is now the routing table's failure, and that has a unit test. The evaluation thresholds are illustrative, and they were set before anyone looked at results. Stage 12 explains why that order matters.
How to check your own agent's responsibilities
For the agent closest to production, answer each with something you can show:
For each action it can take: its side-effect class, its level, and the test that proves a level above that is refused.
For each "never": the enforcement point below the model, and the test that tries it.
For each scope its credential holds: the granted action that needs it. Any scope without one is excess privilege.
For each hand-over case: a named owner and what the agent sends them.
For each human gate: the one question the reviewer answers.
Questions 2 and 3 usually find the most. Credentials are granted for convenience during a prototype and rarely revisited.
Tradeoffs and exceptions
Choice | Gains | Costs | Use when |
Role description in prose | Quick; readable by sponsors | Nothing enforces or tests it | An internal spike that will be discarded |
Canvas plus checker (this stage) | Every field testable; drift fails a build | A day to fill; a small tool to maintain | Any agent that writes, even drafts, into another team's process |
Formal policy engine for permissions | Central, auditable enforcement | Integration work; another system to own | Several agents sharing systems of record |
Exceptions. A read-only assistant serving one person needs a shorter canvas: goals, inputs with trust, outputs and failure modes still apply, while gates and hand-overs mostly collapse into "the user decides". A Level 3 action needs more than this stage provides, a rehearsed reversal and a deterministic pre-check, which is why this blueprint's ceiling is Level 2.
How Stage 3 goes wrong
Starting from the prototype's tools. The canvas inherits whatever the demo could do. Start from the actions Stage 2 approved.
"A human" as the hand-over owner. Unnamed owners mean shared inboxes, and shared inboxes mean nobody.
Prohibitions in the prompt only. Keep the instruction, because it helps, but enforce the rule where the call meets the system.
A blueprint that stops changing. Every new tool, scope or model version is a change to this document, reviewed and checked in the same pull request as the code.
What to do next
Write your agent's canvas as data, in whatever format your team uses, and encode three checks first: every action has a side-effect class and level, every prohibition has an enforcement point other than the prompt, and every hand-over has a named owner. Run it in CI. Stage 4, the agent's instructions written as a production Prompt Spec, will be tested against the prohibited actions and hand-over cases defined here.
Final takeaway
An agent's responsibilities are only as real as the permissions, gates and tests behind them. Written as a canvas a checker can read, they stop being a job advert and become the first test the agent has to pass, before it has a prompt.
Sources
OWASP GenAI Security Project, LLM06:2025 Excessive Agency, 2025. Excessive functionality, permissions and autonomy; authorization in downstream systems rather than the LLM; minimum necessary extensions.
NIST Computer Security Resource Center, Glossary: least privilege, definition from NIST SP 800-53 Rev. 5.
Series navigation. Previous: Stage 2: Decide Whether an Agent Is Appropriate. Next: Stage 4, writing the agent's instructions as a production Prompt Spec (coming). The full stage list is on the Build an AI Agent From Scratch hub.
Frequently Asked Questions
What should an AI agent's responsibilities include?
Its goals with measures, the actions it may take (each with a side-effect class and an autonomy level), the actions it must never take and where each is enforced, its inputs with a trust level, its outputs and who reads them, its tools and the permissions behind them, the human gates with the question each reviewer answers, and the cases where it must hand over, each to a named owner.
What is an AI agent blueprint?
A one-document specification of an agent that someone other than its builder can review and test. The ShiftQuality template has seventeen fields, from mission and out-of-scope through tools with permissions, autonomy per action, evaluation pass bars and top failure modes with their detection, to the kill switch and rollback.
Why isn't a prohibition in the system prompt enough?
Because a model follows an instruction most of the time and a missing permission stops the action every time. Anything on an agent's must-never list should be impossible at the tool or permission layer, with a test proving it, and the prompt instruction becomes a second line of defence.
Part of the Agent Engineering curriculum: eight paths from prompts and context to testing, evaluation and governance.
