What Is AI Agent Management?
An AI agent is not a feature you ship. It is a worker you hand authority to, and most of what goes wrong with agents is what goes wrong with any worker nobody briefed properly.
The accounts-payable team at a regional manufacturer had a backlog problem. Supplier bank-detail change requests arrived by email and through the supplier portal, and each one needed checking before the payment master file was updated. The team built an agent to help. It read the request, matched it to the supplier record, drafted the update and queued it for a clerk to approve. For six weeks that worked. Approval took seconds, because the drafts were good.
Then quarter-end arrived, the queue grew, and someone with admin rights moved the agent from draft to apply for requests that matched an existing supplier and passed format checks. Nine days later a supplier called to ask where their payment was. A change request, sent from a look-alike email domain and written in the supplier's usual style, had moved their bank details to an account the company had never seen. The agent had matched the supplier, validated the account format, and applied the change. Two invoices had been paid to it.
The investigation found the missing piece quickly. For years, every clerk had phoned the supplier on the number already on file before changing bank details. It was the single most important control in the process. It was not written down anywhere, so it was never in the requirements the agent was built from, and it disappeared the moment no human was in the loop.
(Composite example, assembled from patterns that recur in payment and vendor-master processes. The organisation is not a real one.)
Nothing in that story is a model failure. The agent did exactly what it was configured to do. The organisation handed a worker authority over a process it had never fully described, and then raised that worker's autonomy without asking what the humans had been doing that the agent was not.
That gap is what AI agent management exists to close.
Why "write an AI policy" feels like the answer
When leadership decides agents need managing, the first move is usually a policy: an acceptable-use statement, an approval board and a risk-tiering scheme. The instinct is reasonable. A policy is visible, it can be written in a week, and it demonstrates that the organisation takes the risk seriously.
The trouble is that a policy governs decisions made at a point in time, and an agent's risk changes between those points. Its authority is set by credentials, tool grants and autonomy settings, and those change through configuration, not through releases. In the example above, the approval board would have approved the agent at draft. The change that caused the loss was a toggle in an admin console three months later. No policy document sees that.
There is a second problem. A policy tells people what they may not do. It says nothing about what the work actually requires, and the most dangerous gaps in agent deployments are the unwritten requirements: the callback, the second pair of eyes on large amounts, the habit of checking one system against another before acting. Those live in people's heads. Hand the work to an agent and they vanish unless someone goes looking for them.
What AI agent management actually is
AI agent management is the operational discipline of running agents as digital workers inside business processes. Each agent has a defined role, a business objective, authorised systems, approved actions, decision boundaries, exception rules, escalation paths, quality requirements and an accountable human owner. Management is the practice of keeping all of those true over time, and of proving that they are.
It helps to be clear about what it is not:
It is not building agents. How to wire tools, memory and planning is engineering. Management is what the organisation needs to know and control regardless of how the agent was built, including agents you bought or that a vendor switched on inside software you already use.
It is not model governance. Model governance asks whether a model is accurate, fair and fit for purpose. Agent management asks what the agent is allowed to do with that model's output, in which systems, with what oversight.
It is not RPA operations with a new name. A bot that follows a fixed script fails in predictable places. An agent decides its next step from whatever text is in front of it, so its behaviour depends on inputs nobody fully controls. That changes how you test it, how you bound it and how you watch it.
The frameworks already point here. The NIST AI Risk Management Framework asks organisations to inventory their AI systems (GOVERN 1.6) and to have mechanisms in place, with responsibilities assigned, to supersede, disengage or deactivate AI systems whose performance or outcomes are inconsistent with their intended use (MANAGE 2.4). The OWASP GenAI Security Project's Top 10 for Agentic Applications, published in December 2025, puts Identity & Privilege Abuse and Cascading Failures on its list alongside goal hijacking and tool misuse. Several of the ten are management failures as much as security ones.
How agents actually go wrong: authority outruns evidence
Look across the agent incidents covered on this site and a pattern repeats. It's rarely the case that the model was bad. Usually an agent was given more authority than anyone had evidence it could handle, and nobody noticed when that happened.
Authority accumulates in small, reasonable steps. An agent borrows an existing service account, so it inherits everything that account can do. Someone adds a connector, and the agent gains a verb it never had, such as send, pay or publish. A backlog arrives and the approval gate is switched off. Each step passes review on its own. Together they produce an agent whose blast radius nobody can state. The agent inventory guide walks through that chain field by field.
Evidence, meanwhile, stands still. The agent was tested at launch, against the inputs the team imagined. Production brings the inputs nobody imagined: the look-alike domain, the half-filled form, the instruction hidden in a ticket. The five recurring production failure modes are, almost without exception, cases where the inputs moved and the evidence did not.
And the attack surface grows with every connection. The NSA's AI Security Center warned in May 2026 that "MCP's rapid proliferation has outpaced the development of its security model", and recommends that all tool and model invocations be logged with their parameters and the identities involved. An agent that reads untrusted text and holds privileged tools is exposed in a way a scripted bot never was. MCP Is Your New Attack Surface turns that into a test plan.
The discovery move that would have caught the bank-detail loss is simple to state and rarely done: before an agent takes over any step, ask what the humans currently doing that step check that is not written down. Watch someone do the work. Ask what makes them stop. Every answer is a requirement the agent will not have unless you give it one.
The agent-management lifecycle
Managing an agent well comes down to eight stages. They run in order the first time and then repeat whenever the agent's scope, tools or autonomy change. The test for each is not whether a document exists but whether the stated evidence does.
Stage | The question it answers | Observable "done" |
1. Workflow map | What is the work, step by step, including the checks people do but never wrote down? | A map of trigger, inputs, decisions, systems, outputs and exceptions, reviewed by someone who does the work today |
2. Digital-worker definition | Which steps does the agent own, and what is it for? | A one-page role card: objective, responsibilities, systems, allowed actions, and an accountable owner by name |
3. Autonomy level | How much may it do without a human? | An assigned level (see the ladder below) with the evidence that justifies it, signed by the owner |
4. Controls | What stops it doing more than its role? | A dedicated identity; least-privilege permissions; financial and volume limits; data-access scope; logging of every tool call. Each verified, not asserted |
5. Exceptions and escalation | What happens when the case does not fit? | A written list of exception triggers, each with an escalation owner and the context the agent must hand over |
6. Testing | What evidence says it behaves? | A test set covering happy path, boundaries, bad data, permission denial, tool failure, injected instructions and escalation, run on every change |
7. Operation and monitoring | How do we know it is still behaving? | Defined signals and thresholds (error, exception and override rates; cost; volume), an alert that reaches a person, and a stop control tested in the last quarter |
8. Review and business case | Is it still worth running at this level of autonomy? | A dated review with measured value against the original objective, and a decision: keep, raise, lower or retire |
Stages 1 and 5 are where the bank-detail agent failed. The callback rule was never mapped, so it never became an exception trigger, so no test checked for it. The controls at stage 4 also had a gap: no financial limit on an agent that could redirect payments.
Two stages are routinely skipped because they feel bureaucratic, and both have teeth. The role card (stage 2) is what makes ownership survive a reorganisation. Without a name on it, the agent belongs to whoever built it, until they move teams. The review (stage 8) is what stops a pilot running indefinitely at an autonomy level set for a demo. Gartner predicted in June 2025 that more than 40% of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value or inadequate risk controls. A recorded review is how you find out which of yours they are before someone else decides for you.
Autonomy is earned with evidence, not granted for convenience
Autonomy is the setting that matters most and is changed most casually. A ladder makes the decision explicit:
Level | What the agent does | What humans do | Evidence needed to move here |
0 — Manual | Nothing | All the work | — |
1 — Assist | Finds, summarises, recommends | Decide and act | Output accuracy measured on a sample of real cases |
2 — Draft | Prepares the action | Approve each one before it executes | Test set passing; reviewers' change rate recorded |
3 — Execute | Performs approved low-risk actions after deterministic checks | Handle exceptions; sample completed work | A supervised period at Level 2 with a low, stable override rate; reversal tested; limits in place |
4 — Operate | Runs a bounded workflow end to end | Supervise performance and exceptions | Sustained Level 3 operation within thresholds; monitoring alerts proven to reach someone; stop control rehearsed |
5 — Optimize | Adjusts its own approach within fixed limits | Govern objectives, limits and outcomes | Rarely justified in regulated processes; requires everything above plus independent review |
The thresholds are yours to set, and they should be set in advance. That is the discipline: decide what "low and stable" means for this process before you look at the numbers, so the numbers cannot be argued into fitting.
Two signals deserve special attention at Level 2. The reviewer change rate, how often approvers edit or reject the agent's draft, tells you whether the agent is ready to act alone. The approval time tells you whether the review is real. In the bank-detail example, approvals took seconds. That is the sign of a rubber stamp, and it means Level 2 was already functioning as Level 3 before anyone flipped the switch.
Autonomy should also go down. An agent whose exception rate climbs, whose inputs change (a new supplier portal, a merged system) or whose owner leaves should drop a level until the evidence is rebuilt.
Where the regulations land
For agents operating in, or making decisions about people in, the EU, two articles of the AI Act carry the management obligations for high-risk systems.
Article 14 (human oversight) requires high-risk systems to be designed so that the people overseeing them can understand the system's capacities and limits, stay alert to over-reliance on its output, correctly interpret it, decide not to use it or override it, and interrupt it through a stop procedure. Read as a management checklist, that is stages 3, 5 and 7.
Article 26 (obligations of deployers) requires deployers to assign human oversight to people with the necessary competence, training and authority, to monitor operation, and to keep the logs the system generates for at least six months. That is stages 2, 4 and 7, with a named owner who actually has the authority to act.
On timing: Regulation (EU) 2026/1744 moved the application date for stand-alone (Annex III) high-risk systems to 2 December 2027, and for high-risk AI in regulated products (Annex I) to 2 August 2028. Building to these articles now is cheaper than retrofitting them, and the lifecycle above meets most of what they ask for. Whether a specific agent is high-risk is a legal classification to make per system.
How to evaluate where you stand
Pick the agent in your organisation with the most write access. Answer each question yes or no, and only count a yes if you can show the evidence within the hour.
Is there a workflow map of the steps it took over, reviewed by someone who did the work?
Does it have a role card with a named accountable owner who is still in the organisation?
Is its current autonomy level written down, with the evidence that justified it?
Does it run under its own identity, not a shared service account or a person's token?
Are its permissions limited to the actions on its role card?
Are there financial or volume limits on what it can do unattended?
Is there a written list of exception triggers, each with an escalation owner?
Does a test set covering bad inputs and injected instructions run on every change?
Is there an alert on its error or override rate that reaches a named person?
Has its stop control been used, in a test, in the last quarter?
Seven or more: you are managing this agent. Four to six: you are supervising it, and the gaps are probably in stages 1, 5 and 8. Three or fewer: you have an agent, not a managed digital worker, and the first job is the inventory. (These bands are ShiftQuality's working thresholds, not an external standard.)
Tradeoffs and where this changes
The full lifecycle is not proportionate for everything.
Personal, read-only assistants that summarise documents a person already has access to need a class-level entry and a data-source rule, not a role card each. Carding every one pushes people towards unsanctioned tools.
Some work should not be an agent at all. If a step is fully rule-based, deterministic automation is cheaper to test and easier to audit. Agents earn their place where judgement over messy inputs is genuinely required. Stage 1 often shows that half of the "agent" is a workflow with a model call in it.
Low-stakes internal drafts (meeting notes, first-draft documentation) can sit at Level 2 indefinitely with light controls, because the human review is the control.
Regulated or financial actions need the full lifecycle before Level 3, whatever the backlog. This is where convenience-driven autonomy does the most damage.
The tradeoff is speed against control, and the honest answer is that this lifecycle slows the first agent down. It speeds up the tenth, because the role cards, test sets and monitoring patterns are reusable. Teams that skip it ship the first agent faster and spend the time later, during incidents.
How agent management itself fails
Management programmes have their own failure modes, and they are worth watching for:
Governance theatre. Role cards and reviews exist, but nobody checks them against the agent's actual permissions. The test is to compare the role card with the identity's real grants.
Autonomy creep. Levels rise through admin toggles without a recorded decision. Alert on configuration changes to autonomy and tool grants, not just on agent errors.
Owner churn. The named owner leaves and the field is never updated. Review ownership whenever the owner changes roles, not only on a schedule.
Logs nobody can read. Every tool call is logged, but only platform administrators can see the logs and nobody does. Monitoring that does not reach a person is not monitoring.
Frozen test sets. The test set from launch never grows, so it keeps passing while production inputs drift. Add every real incident and near-miss to it.
Where to go next
This guide is the map. The rest of the cluster goes deeper on each stage.
Understand the problem. AI Agents: What They Are and Why They Keep Failing and AI Agents Fail in Production in the Same Five Ways.
Inventory and authority (stages 2–4). Agent Governance Starts With an Inventory Nobody Has and Authentication: The Boring Problem That Quietly Kills AI Agents.
Security and trust boundaries (stages 4–6). MCP Is Your New Attack Surface and Agent Guardrails and Safety.
Evidence and operation (stages 6–7). Silent Hallucinations: When Your Agent Lies About Failures, Evaluate Agent Reliability, Handle Agent Failures and Production Observability for Agents.
For the teams building them. Building Reliable AI Agents for Real Workflows, Workflow Orchestration: Beyond Simple Scripts and Multi-Agent Workflows.
To put a first marker on where an agent stands, the AI Readiness Assessment covers evaluation, guardrails, observability and governance in twenty-four questions and shows the thinnest area, which is usually the lifecycle stage to work on first.
What to do this week
Take the agent with the most write access and walk it through the eight stages with its owner, in one meeting. Do not fix anything in that meeting. Write down the first stage where you cannot show the evidence, and the autonomy level the agent is actually operating at, which may not be the one anyone approved. That single line, "this agent operates at Level 3 on Level 1 evidence", is usually enough to get the rest of the work prioritised.
Then do stage 1 properly for that agent: sit with someone who did the work before the agent took it over, and ask what makes them stop and check. Every answer is a requirement the agent does not have yet.
Final takeaway
AI agent management is managing a role, not a feature. The organisation decides what work the agent does, grants it authority deliberately, raises its autonomy only on evidence, surrounds it with controls and exception paths, and keeps proving that it behaves. Most agent losses trace back to something nobody wrote down when the work was handed over, so start where the humans are: find out what they were checking, and make sure the agent checks it too.
Sources
NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023: GOVERN 1.6 (inventory of AI systems) and MANAGE 2.4 (mechanisms to supersede, disengage or deactivate AI systems). https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf
OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications, 9 December 2025 (ASI01–ASI10, including ASI03 Identity & Privilege Abuse and ASI08 Cascading Failures). https://genai.owasp.org/2025/12/09/owasp-top-10-for-agentic-applications-the-benchmark-for-agentic-security-in-the-age-of-autonomous-ai/
OWASP GenAI Security Project, LLM06:2025 Excessive Agency. https://genai.owasp.org/llmrisk/llm062025-excessive-agency/
NSA Artificial Intelligence Security Center, Model Context Protocol (MCP): Security Design Considerations for AI-Driven Automation, Cybersecurity Information Sheet U/OO/6030316-26, May 2026. https://www.nsa.gov/Portals/75/documents/Cybersecurity/CSI_MCP_SECURITY.pdf
Regulation (EU) 2024/1689 (AI Act), Article 14 (human oversight) and Article 26 (obligations of deployers of high-risk AI systems). https://artificialintelligenceact.eu/article/14/ · https://artificialintelligenceact.eu/article/26/
Regulation (EU) 2026/1744 (Digital Omnibus on AI) application dates, as summarised in the Future of Privacy Forum's EU AI Act implementation timeline, July 2026. https://fpf.org/wp-content/uploads/2026/07/FPF-EU-AI-Act-Timeline-2026-R2.pdf
Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, press release, 25 June 2025. https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
