User Acceptance Testing (UAT): How to Run It Well

The claims-intake tool passed UAT on a Thursday. Eight adjusters had worked through a scripted scenario in a conference room, checked their boxes, and signed. The room felt good. The project manager forwarded the signed sheet to the sponsor with a note: "Users love it."

Six days after go-live, the same eight adjusters filed twenty-two tickets. Not about crashes — the software did exactly what the spec said. The tickets were about the one screen where you had to save a claim before you could attach a photo, which meant that every adjuster standing in a driveway with a policyholder had to key the whole claim, save, wait, then come back for the photos. In the conference room, with clean test data and no policyholder watching, nobody had felt that friction. They had followed the script, which said "enter claim details, then click Save, then attach supporting documents." The script hid the problem because the script was the problem.

That gap — between a system that meets its specification and a system that works for the people who use it — is the entire reason UAT exists. And it is exactly the gap that a "looks great" sign-off is engineered to paper over. This piece is about running UAT so that it catches the twenty-two tickets before launch, not after.

What UAT is actually checking

Internal testing answers one question: does the system do what we said it would? Unit tests, integration tests, and end-to-end tests all measure the build against the specification. They are essential, and they are not enough, because a system can implement a wrong specification perfectly. It will pass every internal test and still fail the people it was built for.

UAT checks the other direction: does the specification meet the users? It is the last point in the delivery pipeline where a human who does the actual job can say "this is not how the work happens" before the software becomes the way the work happens. If your team never validated the real workflow during discovery — how an adjuster works in a driveway, not in a conference room — UAT is where that skipped discovery comes due. You cannot test your way to a feature you never scoped correctly; UAT can only reveal the gap, not close it for free.

That reframing changes who you invite and what you watch for, so get it explicit before you schedule anything. Write one sentence at the top of your UAT plan: "This session succeeds if it surfaces workflow problems our internal tests structurally cannot catch." If your plan instead reads "confirm the system works," you are booking a demo. (For the upstream half of this — writing criteria that are testable in the first place — see Acceptance Criteria: Given/When/Then Done Right.)

Why the demo version is so tempting

Nobody sets out to run UAT theater. It happens because every incentive in the room points that way. The project is late, the sign-off unblocks the launch, the participants are busy people doing you a favor, and the polished build genuinely looks finished. Asking a room of stakeholders to hunt for problems in something that looks done feels almost rude — so they validate that it looks done, which is not the same thing.

The scripted scenario makes it worse. A good script walks users down the happy path the team already tested. If the script and the internal test suite cover the same ground, UAT becomes a slower, more expensive re-run of tests that already passed. The findings you actually need — the driveway problem — live off the script, in the friction of real use, and a pure-script session never goes there.

Before your next UAT, run one check on the script itself: for each step, ask "would our end-to-end tests already fail if this were broken?" Every step where the answer is yes is a step that will teach you nothing new. Keep a few for grounding and orientation, then spend the rest of the session somewhere the tests can't reach. (If you're not sure your internal coverage is solid enough to lean on this way, Testing Fundamentals: Why We Test is the prerequisite.)

The discovery move: watch the hands, not the mouth

Here is the technique that separates a useful UAT from a signature-collection exercise, and it is almost embarrassingly simple: watch where users hesitate, and treat every hesitation as a finding.

Users are polite. Asked "is this working for you?", a participant who is quietly confused will say "yeah, this is fine" — because they assume the confusion is their fault, because they don't want to insult the team that built it, because saying "fine" ends the meeting faster. Their words are the least reliable signal in the room. Their hands are the most reliable. A three-second pause before clicking, a cursor that drifts to the wrong region first, a scroll up to re-read a label, an "wait, where's the…" that they answer themselves — each is a place the design fought the user, and each is invisible in a pass/fail script result.

So the discovery move is a diagnostic posture, not a question you ask. You set up the session so you can observe the work, and you write down friction the participant never verbalizes. The single most useful prompt is not "does this work?" but "show me how you'd do this on a real Tuesday" — and then you go quiet and watch. The silences are the data.

This week, add one column to your UAT note-taking sheet labeled "Hesitations." Every time a participant pauses, backtracks, or asks a question they then answer themselves, log the timestamp and what they were trying to do. Review that column at the end — it is your real defect list, and it will be longer than the one participants tell you about.

A UAT session, end to end

(Developed example — composite scenario.)

A regional insurer is replacing a claims-intake tool. Internal testing is green: every acceptance criterion passes, the end-to-end suite is clean. The team schedules a half-day UAT.

Participant selection. The product owner's first instinct is to invite the three team leads who attended the design reviews. The QA lead pushes back with a single question: "Which of these people will personally key a claim next Tuesday?" The leads won't — they approve claims, they don't file them. So the session is rebuilt around five field adjusters who file claims daily, plus one adjuster from a rural territory with spotty connectivity, because "works on a laptop in a driveway" is a real condition the spec never named. Six participants, all close to the actual use.

Structure. The session is split. The first forty-five minutes are scripted: two critical paths — file a routine auto claim, and file a claim with photo attachments — with expected outcomes written down, so everyone touches the core flow and the team has a clean baseline. The remaining ninety minutes are exploratory: "file the last three claims you actually handled, the way you'd handle them." No script. The facilitator shares each adjuster's screen and takes notes in two columns — one for stated issues, one for hesitations. (This exploratory half is a discipline, not free time; Exploratory Testing: A Discipline, Not a Hack is where that structure comes from.)

The moment. During the scripted photo path, everything passes — the script says save first, then attach, and it works. But in the exploratory block, the rural adjuster is recreating a real total-loss claim. She fills in the vehicle details, then stops. Her cursor hovers over the top of the screen, drifts toward a "camera" area that isn't there, and she says, half to herself, "okay so I'd normally snap the photos first while I'm looking at the car…" Then she finds the Save button, saves, waits on her throttled connection, and comes back to attach. Three seconds of hesitation. She never complains. On the sheet, she'd have checked "pass."

Reading the moment. The facilitator flags it and probes gently: "walk me through how that goes on-site." It comes out that the enforced order — save the claim, then attach photos — inverts how adjusters actually work: photos first, while the damage is in front of them, before they've keyed anything. This is not a bug. The system does precisely what the spec required. It is a design issue: the spec encoded the office workflow, not the driveway workflow. Because it surfaced in UAT and not in a support queue, it is the single most valuable finding of the day.

Capture. It gets written as a specific, actionable report — scenario, expected, actual, impact, frequency — not as "photo thing is awkward." Roughly: "Adjusters attach photos before keying claim details (photos taken on-site first). Current flow forces claim-save before attachment, blocking the natural order. Affects every field-filed claim with photos — majority of volume. Classify: design issue, not defect." (What makes that write-up usable rather than vague is the subject of Bug Report Template: What Makes One Useful.)

Sign-off decision. Five of six critical paths are clean. The photo-order issue is real and high-frequency but not a code defect, so the team doesn't block launch on a full redesign. Instead the sign-off is conditional and explicit: launch proceeds with a fast-follow that adds a "start with photos" entry point, the affected adjusters are told it's coming and why, and the design issue is logged as accepted-with-remediation rather than silently shipped. Nobody signs a blank "looks great." Everybody signs something true.

Bug or design issue: sort every finding before you triage

The photo-order problem teaches the most important sorting move in UAT. A finding is either a bug — the system violates its specification — or a design issue — the system honors its specification but the specification was wrong. They look identical in the room ("this is awkward") and demand completely different responses. Mislabel them and you either "fix" working code or dismiss a genuine defect as opinion.

Dimension	Bug	Design issue
What's wrong	System violates the spec	System matches the spec; the spec is wrong
Where it originated	Build / implementation	Discovery — a workflow or condition never scoped
Typical signal in UAT	"It errored," "it lost my data"	Hesitation, workarounds, "I'd normally do X first"
Correct owner	Engineering fixes the code	Product re-decides, then engineering builds
Can it block launch?	Often yes, if on a critical path	Judgment call — weigh frequency and workaround cost
Why UAT is uniquely good at it	Sometimes; e2e tests catch many first	This is UAT's whole reason to exist

The practical consequence: design issues route to whoever owns the specification, not to the bug tracker's default engineering queue, and they carry a decision (accept, remediate, or block) rather than just a fix. Note too that the boundary you're sorting against is your Definition of Done versus your acceptance criteria — two different contracts that UAT quietly tests at once (Definition of Done vs Acceptance Criteria draws that line).

For your next session, label every finding "bug" or "design issue" in real time, in a visible column. At the end, count the design issues. A UAT that produced zero design issues didn't validate the specification — it re-ran your integration tests with more people in the room.

Where UAT sits, and where it doesn't reach

UAT is one instrument, tuned for one job, and teams get burned by asking it to do a neighbor's work. It is not beta testing and it is not a controlled rollout, and running the wrong one for your risk is its own failure mode.

• UAT — a small set of real users run defined scenarios plus real work against a release candidate, with a formal, explicit sign-off, before launch. Best when a wrong workflow assumption is expensive to discover in production.

• Beta — a broader, self-selected group uses the system in the wild over time; feedback drives iteration, not a go/no-go gate. Best when you need scale, variety, and load you can't manufacture in a room.

• Controlled rollout (canary / phased) — the real system ships to a small slice of production traffic with real stakes and fast rollback. Best when the risk is operational (performance, integration, edge-case volume) rather than "did we understand the workflow."

Sequence them: internal testing → UAT → beta → controlled rollout → general availability. Each catches a class the others structurally can't. Before your next release, name the one failure you're most afraid of. If it's "we built the wrong workflow," you need UAT and no amount of beta traffic substitutes. If it's "it falls over at 10× load," UAT with six people in a room will never find it — that's a rollout question.

Making sign-off mean something

A signature is a data point about a signature. What you actually want to know is whether real users, having genuinely exercised the system against real work, believe it serves the job — and whether every finding has an owner and a decision. Sign-off should certify three things: participants actually tested (not watched a demo), findings are either resolved or explicitly accepted with a named owner, and the people who do the work believe it works for the work. Anything less is a formality dressed as a gate.

The failure modes are predictable, so pre-empt them. Participants who won't engage need scenarios and an observer, not a "go ahead and click around." Sign-off pressure needs budgeted fix-time after UAT baked into the schedule, so "we found things" doesn't collide with "we ship Friday." Wrong participants need the Tuesday question asked before invites go out. And "no follow-up" — findings that evaporate after the meeting — needs every item tracked to a closure decision, accepted or fixed, with a name on it.

Run this check on your last sign-off sheet: for every finding, can you name the person who owns its resolution and point to the decision that was made? If any finding traces to nobody, your sign-off certified attendance, not acceptance — and that is the exact shape of a signature that ships the driveway problem.

Sources

• The claims-intake opening and the half-day UAT walkthrough are illustrative composite scenarios built from common patterns in software delivery, not accounts of a specific named organization or client. They are labeled as such in the text and are used to demonstrate the technique, not as evidence.

• No external statistics are cited in this article. Claims about how UAT, beta testing, and controlled rollouts differ are definitional and drawn from standard software-delivery practice; where the piece asserts a judgment (for example, that hesitation is a more reliable signal than stated approval), it is presented as the author's interpretation, not as a measured finding.

Keep learning. This article is part of the Software Testing Foundations path in the ShiftQuality Learning Center.