Acceptance Criteria for Non-Deterministic Features

Given-When-Then breaks on a feature that answers differently every time. The repair is not better wording — it is sorting each criterion by how many runs it would take to prove it false.

Illustrative composite: Penhale Mutual is ShiftQuality's running case. Its figures are illustrative, not measured.

Penhale Mutual's claims-triage assistant reads a first-notice-of-loss report and drafts a summary plus a recommended handling path. It went into sign-off with eleven acceptance criteria and sat there nine days.

The demo was fine, the eval suite ran green, the adjusters wanted it. What nobody could do was say whether it had passed — and the argument was never about the assistant's output, but about what the sentence on the card asked anyone to check. Sorting the eleven took an hour, on one question.

The unit is one criterion, and the question that sorts it

This post owns one narrow thing: a single criterion, and what one run is allowed to prove about it. One line on the card.

The neighbouring questions belong to the neighbouring posts: the definition of done sets what holds for every AI feature; sample size settles how many runs a percentage needs; eval calibration settles whether the score is real. None is fixed by rewording a criterion, yet all three land in sign-off when the criteria have not been sorted.

The sorting question: how many runs would it take to prove this criterion false? One run → an invariant. Many runs → distributional. No number of runs → not a criterion at all, but a preference someone will defend in a meeting. Get the shape wrong and you have not written a weak criterion but one that licenses the failure it looks like it prevents.

The test: take three criteria off your current AI story and ask, for each, how many runs would prove it false. Any criterion answered "none" is the one stalling your sign-off.

Invariants: a percentage on a yes/no property is a permission slip

Penhale's card said: ≥95% of answers cite the policy number.

It reads like rigour. But one run falsifies it: a single summary with no policy number is a defect a regulator opens a file about. The property is binary, and a binary property one run can kill is an invariant.

So what is the 95% doing? Authorising the other 5%. At roughly a hundred triaged claims a week, that is five regulated outputs with no traceable source, at a rate everyone in the approval chain signed — and the uncited ones are the only outputs anybody ever asks about.

The bar landed there because it was a measurement, not a judgement: somebody ran the eval set, observed 95%, and wrote it down as the target. A sub-100% bar on a binary property is a confession that no mechanism enforces it — no post-generation check, no schema validation, no refusal path — converted into an approved rate. The repair is the missing check. Where the check cannot itself be exact — deciding that an amount is "absent from the file" may need a model of its own — the 100% is held by a refusal path or a human review step instead. Either way it is enforced, not averaged.

Invariants — one violation fails the story:
- The response cites the policy number it used
- The response states no amount, date or claim number absent from the file
- Where the file cannot answer, the assistant says so rather than inferring

The test: for every percentage criterion, ask whether one violating output would be a defect somebody would file. If yes, the number is 100 — and you have found the enforcement mechanism nobody built.

Distributional: name the set before you name the number

The inverse mistake is better disguised: 100% on a judgment property. Demanding it on open-ended summarisation either blocks the story forever or teaches everyone to soften the rubric until it passes, and then the pass rate moves for reasons nobody logged.

Here are five of the eleven, one for each way a criterion goes wrong. The last column is the one to read twice.

Criterion as written	Runs to falsify	Shape it should have been	What the mis-shape licensed
"≥95% of answers cite the policy number"	1	Invariant, 100%	1 in 20 regulated outputs with no traceable source
"≥98% of answers contain no amount absent from the file"	1	Invariant, 100%	Fabricated figures, at an agreed rate
"≥90% of answers are correct"	many	Distributional — but on which set?	A pass rate with no defined denominator
"Responses are helpful and professional"	none	Not a criterion — a rubric item or nothing	An argument in the sign-off meeting
"Handles ambiguous claims gracefully"	none	Boundary — needs a named situation	Everyone assumed someone else had checked

Three of the eleven were invariants wearing percentages; two were adjectives no run could falsify. But the finding that broke the deadlock was a count: zero of the eleven said which eval set the percentage was measured on. Ninety per cent of answers are correct — of what? A percentage with no denominator is a number about nothing; nine days of stalling was an accurate reading of it.

The set existed. Discovery had produced hand-written ideal outputs months earlier; nobody had connected them to the card. A distributional criterion must name the set, its size, its version and its hard subset — the set is a sample somebody chose, and the choosing is where the number comes from. How large it must be is separate arithmetic.

The test: read your distributional criteria aloud and stop at the word "of". If what follows is not a specific, versioned set with a case count, the percentage has no denominator.

Boundary: Given-When-Then survives, on categories

The tail — the unanswerable question, the ambiguous claim, the endpoint that is down — is where a confident system stays confident. It is also where Given-When-Then still works, because a boundary criterion specifies a category of situation and response, not one exact string.

Given a claim the FNOL file does not describe well enough to route
When the assistant responds
Then it names what is missing and routes to a human — it does not pick a path

"Handles ambiguous claims gracefully" failed the sorting question because gracefully names neither a situation nor a response. Rewritten as above, one run can contradict it.

The test: count the criteria that describe the feature refusing, failing or handing off. If the count is zero, you have specified the demo rather than the feature.

The rubric is part of the criterion, not a detail of it

Every distributional criterion rests on three words doing the load-bearing work: rated correct by. If the rubric behind them is not written down, you have a different criterion per reader, and the pass rate becomes a property of who reviewed.

The mechanism that catches this is a reproducibility check, deliberately not an accuracy check. Two people score the same ten outputs independently, without conferring. Whether their scores match reality is a calibration problem belonging to the eval layer; the question here is narrower — does the criterion mean the same thing to two readers? If not, re-running the eval next sprint with a different reviewer moves the number, and nobody can say whether the feature changed.

The test: have two people score the same ten outputs independently against the rubric. More than two disagreements and the rubric is the defect — fix it before anyone argues about the model.

When this is the wrong move

Some stories do not repay the sorting:

• A spike. If the acceptance criterion is honestly "we learned whether this is possible," write that down and stop. Shaping criteria around the question you are asking because you cannot answer it is theatre.

• A feature a deterministic check already covers. If a schema validation and a lookup settle whether the model classified a document correctly, you have a normal criterion; the AI is an implementation detail.

• A prototype before anyone has read the outputs. In the first fortnight a rubric mostly encodes a guess. Read fifty real outputs, then write the criteria — in that order.

The test: ask what happens if this story fails sign-off. If the work is discarded either way, the criteria document a decision nobody is making — if something reaches a customer, they do not.

To write the requirement itself, the AI Feature Requirements Worksheet walks one feature through ten moves: hard and soft properties, pass-rate thresholds, failure-mode bounds, drift monitoring and audit logging.

What to change this week

Take one live AI story and re-sort its criteria by runs-to-falsify: an hour, one column per shape, and a fourth for the ones that fit nowhere. Expect it to delete more than it adds.

Penhale's eleven came out as eleven again, differently shaped. The three disguised invariants went to 100%, and a fourth was pulled out of the "correct" criterion, where it had been hiding: where the file cannot answer, say so. Three distributional bars now each name the 60-case set and its 18-case hard subset. The two adjectives were deleted rather than rescued — nobody could name a situation they described, and that was the finding — and four boundary cases took their place: unanswerable, ambiguous, missing document, endpoint down. It was signed off the same afternoon. The nine days had never been a disagreement about the assistant — they were five people reading one sentence and checking five different things.

Sources

• Dan North, "Introducing BDD", Better Software, March 2006 — the origin of the Given/When/Then scenario form for acceptance criteria that this post adapts.

• Penhale Mutual, its eleven criteria, the nine-day stall and all figures in it are an illustrative composite, not a record of a specific team.