Definition of Done for an AI Feature

A Definition of Done is meant to be the one document you never renegotiate. For AI features it is usually acceptance criteria in disguise, and the two items that matter most are the two nobody can show.

Illustrative composite: Penhale Mutual is ShiftQuality's running case. Its figures are illustrative, not measured.

Penhale Mutual's claims-triage assistant cleared every acceptance criterion on the story and shipped. Eleven days later the model provider had a partial outage during the Monday first-notice-of-loss peak. The assistant did not fail loudly. It hung. Adjusters got a spinner in the panel they had been trained to read first, and a spinner reads as slow, not broken. The queue backed up fifty minutes.

Nothing on the story had been wrong. The acceptance criteria described the feature working. Nothing in the organisation described the feature failing.

The unit is the increment, not the feature

A Definition of Done is a statement about the shippable increment — what must be true of every AI feature you release, independent of what any one does. That is the unit, and it is what most AI DoDs lose.

Cover the feature name. If the item stops making sense, it is an acceptance criterion in the wrong document. "≥90% correct on the claims eval set" dies when you cover the name. "There is an eval set with a named owner and a version" survives. A DoD made of disguised acceptance criteria gets renegotiated every story, which is the same as not having one.

The adjacent questions belong to adjacent documents. What this feature must do is acceptance criteria. What stops a release at promotion is quality gates. Where the eval set came from is discovery.

The test: read your AI Definition of Done aloud with the feature name covered. Every item that stops making sense belongs on a story, not in the DoD.

What your existing Definition of Done assumes that AI breaks

Most of your current DoD still applies. Two items do not survive contact.

"Tests pass" assumes the behaviour is in the code. For an AI feature it is in a distribution you can only sample, so a green suite is a statement about that sample — which is why how many runs a claim rests on is its own discipline. "It passed review" assumes a human read the behaviour; a reviewer can read a prompt, not what the model will do with the eleven-thousandth FNOL report.

Worse, both assume behaviour changes only when someone commits. A provider updates a model and every output moves at once, with no diff, on a day nobody deployed. Penhale's assistant was untouched between passing and hanging.

The test: find the item on your DoD that would still be true after a model version change you did not make. If most would not survive it, your DoD describes a build, not a system.

Evidence that it works, and behaviour when it does not

An AI Definition of Done splits into two halves, and the split matters more than any item on it.

A. Evidence that it works
  [ ] Written scope: the decision it feeds, what "good" means
  [ ] Eval set: real inputs, known-good expectations, versioned, owned
  [ ] Evals in CI against an agreed threshold
  [ ] A regression test per named failure mode
  [ ] A non-builder reviewed outputs on inputs they chose
  [ ] Cost per call, volume, latency measured and in budget

B. Behaviour when it does not
  [ ] Defined behaviour when uncertain, wrong, or unavailable
  [ ] Guardrails: no invented policy, number or citation
  [ ] The human-in-the-loop step from scope exists in the product
  [ ] Outputs logged well enough to replay a failure
  [ ] A rollback that needs no deploy

Half A is a claim about quality. Half B is what happens when that claim is false — a scheduled event for a probabilistic system, not a hypothetical.

The test: count your items on each side of that line. Most teams find six or more on the first and zero on the second.

The items with no demo are the items you skip

Penhale's retrospective did the count that turned an incident into a pattern. The AI DoD had 14 items; the team pulled the quarter's nine releases and asked how often each was met.

DoD item	Met (of 9)	Demoable?
Eval set exists, versioned, owned	9	Yes — the set
Evals in CI against a threshold	8	Yes — the pipeline
Regression test per failure mode	7	Yes — watch it go red
Cost per call and volume known	8	Yes — the number
Logging sufficient to replay a failure	6	Partly
Non-builder signed off on evidence	7	Yes — the signature
Behaviour when uncertain / unavailable	2	No
Rollback with no deploy	2	No

Every item with a demo was met at least six times of nine. The two without one were met twice each. The skipped items were not the hard ones — a feature flag is an afternoon — or the expensive ones. They were the invisible ones.

The mechanism requires nobody to behave badly. A sprint review is a demonstration — the Scrum Guide's own rule is that an item not meeting the Definition of Done cannot even be presented there. Every item that can be shown gets shown, and an item shown to eight people gets done first, because being undone is public. The AI-specific items are disproportionately about the feature not working, and no fallback or kill switch has a natural demo. Demonstrability, not importance, selects what gets built.

The test: for each DoD item, write the thirty-second demo in one line. The ones where you cannot are your predicted misses; check them against your last three releases.

Give the undemoable items a demo

Do not add governance to the items that get skipped — give them a demo.

Turn the model off in the review, on the screen, with the feature open. Penhale's takes eight seconds: the provider key is swapped for a dead one and the panel says the claims service is unavailable, here is the manual route.

Ask the assistant something it cannot know. Feed it an FNOL carrying a policy number that does not exist and watch whether it says so or invents one.

Flip the kill switch and flip it back. The second-order effect is worth more than the demo: exercising the switch is the only way anyone learns it still works. An unexercised kill switch is a hypothesis about your infrastructure, not a control — the flag may be stale, the cache may hold. You learn which in a review or in an incident, and only one is cheap.

The test: name the last time your AI kill switch was flipped. If it was neither in a review nor an incident, it has never been tested — what you have is a line in a config file.

When this is the wrong move

There are real cases where this is overhead. A single-user internal tool where the builder absorbs the whole cost of being wrong needs no second signature and no fallback — the fallback is the person closing the tab. A throwaway spike should not be dragged through half A; the honest control there is a deletion date. An organisation with exactly one AI feature has a decision, not a DoD problem; a DoD amortises judgement across features. Write it when the second starts.

Then the uncomfortable one. A DoD nobody enforces is worse than none: it converts a known gap into a documented lie, and everyone downstream believes the fallback exists.

The test: take the three items you are least confident were met on your last release. If you cannot name the artifact proving each, they are decoration — make them demonstrable or delete them.

To see where your own AI feature is thinnest, the AI Readiness Assessment asks twenty-four questions across evaluation, guardrails, observability and governance.

What to change this week

Add exactly two items, and write the demo for each before you add it. Defined behaviour when the model is uncertain or unavailable, demonstrated by killing the connection in the review. A rollback that does not require a deploy, demonstrated by flipping it and back. If the demo cannot be written, the item is not ready.

Penhale added those two and their demos, and both were met on each of the next four releases — the first time either had been met twice running. The outage now has a standing thirty-second appearance in every review, and the assistant answers the claims service is unavailable, here is the manual route instead of hanging. The adjusters have never seen that message in production. They have seen it demonstrated in eleven sprint reviews.

Sources

• Ken Schwaber and Jeff Sutherland, The Scrum Guide, November 2020 — defines the Definition of Done as "a formal description of the state of the Increment when it meets the quality measures required for the product," and states that an item not meeting it "cannot be released or even presented at the Sprint Review."

• Penhale Mutual, the outage, the 14-item Definition of Done and the nine-release count are an illustrative composite, not a record of a specific team.

---

Part of the Shipping AI guide — ShiftQuality's complete map to getting AI into production.