Exploratory Testing: A Discipline, Not a Hack

Shawn West
Mar 9
9 min read

Updated: Aug 10

The scripted suite was all green the morning a tester found a serious data-leak bug in twenty minutes — by chasing a hunch the script was never written to have. That gap is the entire case for exploratory testing. Not clicking around: a discipline with a charter, notes, and evidence, aimed squarely at the bugs that live where nobody thought to look.

The build was green. Two hundred and forty automated checks, every one passing, the release dashboard a wall of ticks. By the numbers, the workspace-switching feature was done. Then a tester opened a session with a one-line charter — explore workspace switching for data that leaks across the boundary — and inside twenty minutes filed a P1: switch workspaces while a heavy report is still loading, and for a beat the previous workspace's data paints on screen before the new workspace's data replaces it. In a multi-tenant product, that is one customer briefly seeing another customer's numbers.

Every scripted test had passed because every scripted test checked what someone had already thought to check. Load workspace A: correct. Load workspace B: correct. Switch A to B: correct. Nobody had written switch while A is still mid-load, because nobody had thought of it — and a script can only assert the cases its author imagined. The bug lived in the gap between the checks, which is exactly where bugs live. Finding it was not luck and it was not clicking around. It was a tester following a discipline built to surface the thing no one scoped.

That is the reputation problem exploratory testing has to shake. To people who don't do it, it looks like unscripted flailing. Done well it is the opposite: the most structured way a team has to go looking for its unknown-unknowns.

Why a script can't find what exploration finds

Start with what a scripted test actually is. It is a decision, frozen. Someone sat down, thought about how the feature should behave, and encoded those expectations as steps and assertions. That is genuinely valuable — it pins behavior down so it can't regress, and that repeatable safety net is why we test at all. But it carries a structural limit that no amount of coverage erases: a script can only check the cases its author already imagined. It is a memory of yesterday's understanding, run again today.

Exploration is built for the opposite job. The exploratory tester is learning the system, designing tests, and running them at the same time — Cem Kaner's definition, and the three happening together is the whole point. What they observe in one moment reshapes what they try in the next. Loading feels slow, so they try switching mid-load. A field accepts an emoji, so they wonder what the export does with it. Each result is a new hypothesis. The script walks a fixed path; the explorer walks toward the smell.

This maps straight onto the discovery-first idea that runs under everything we write: projects fail in discovery, not in code, and the bug lives where nobody looked. A scripted suite is a map of the territory the team has already discovered. Exploration is how you walk off the edge of the map on purpose. The workspace-leak bug wasn't a coding failure — the code did exactly what someone specified. It was a discovery failure: the mid-load race condition was never scoped, so it was never checked, so it shipped green. You cannot script your way to a case you never conceived. You can explore your way to it.

Which is why the two are partners, not rivals. Scripts hold the known ground. Exploration takes new ground and hands it back to the scripts to hold — a rhythm we walk through in detail in testing that catches real bugs. Neither replaces the other, and a team running only one is leaving the other half of its bugs in the field.

One session, start to finish

Enough principle. Here is what the discipline actually looks like when a tester sits down and does it — a single session walked end to end, so the structure and the heuristics earn their place by being used, not listed.

(Developed example — composite scenario.)

The charter. Maya has ninety minutes before the release candidate locks. She does not open the app and start poking. She writes one sentence in her session sheet:

Explore workspace switching under slow and interrupted conditions, looking for data that crosses the tenant boundary. Time-box: 90 min. Build: RC-2.14.

That sentence does real work. It is narrow enough to focus a mission — the tenant boundary under interruption — and wide enough to leave room to be surprised. A charter like "test the app" would have dissolved into aimless clicking; a charter like "verify switching from A to B works" would have just re-run the script. The charter names a place to hunt and a reason to hunt there. A good one is a hypothesis about where bugs hide, not a task list.

The time-box and the notes. She starts a timer and starts a running log — not formal test cases, a record of investigation. Every few minutes a line lands in one of five buckets: test ideas tried, bugs, concerns (something felt off), questions (needs a real answer), and areas not covered (so the gap is visible, not silent). The log is the deliverable. If it isn't written down, the session didn't happen.

The moves, and the one that paid off. She reaches for a heuristic to generate ideas rather than waiting for inspiration. The one that fits a boundary-crossing charter is consistency — is the system's behavior consistent with what a user would reasonably expect? She runs the clean cases first: switch A to B, correct; switch back, correct; switch with each on a slow throttle, correct. Green, green, green — the same ground the script already held.

Then the hunch. Throttling the network, she notices Workspace A's dashboard takes a full four seconds to render its charts. Four seconds is a window — a state the system sits in, not just passes through. The "what if?" writes itself: what if I don't wait? What if I switch to B while A is still painting? That is the move a script never makes, because a script waits for the page to settle before it asserts; the interesting state is the one the assertion skips past.

She throttles the connection, loads A's heaviest dashboard, and at second two — mid-render — clicks into Workspace B. For roughly one second, B's frame is on screen with A's revenue chart still inside it. Then A's data blanks and B's paints in. Her log, verbatim:

BUG (P1): Switch to Workspace B while A's dashboard is mid-load → A's revenue chart renders inside B's shell for ~1s before replacing. Cross-tenant data exposure. Repro 4/5 on 3G throttle; not repro on fast connection. Cause guess: async fetch from A resolves after the view swaps and paints into the new context. Screen recording attached.

Notice what made that line useful: a real repro path, a rough reliability count (4 of 5), the exact condition (throttled connection), and a mechanism guess for whoever picks it up. That is the difference between a note that becomes a fix and a note that becomes a shrug.

The debrief. At the ninety-minute mark she stops — the time-box is a discipline, not a suggestion — and spends five minutes closing the session: one P1 filed, two smaller concerns logged, one open question for the product owner (should a slow switch show a loading state at all?), and one honest gap — she never got to permission changes mid-session, so that charter is still owed. Then the P1 becomes a proper defect write-up, because a note in a session sheet and a report an engineer can act on are not the same artifact; what separates them is everything that makes a bug report actually useful.

Total elapsed on the finding: about twenty minutes. Nothing mystical happened. A charter aimed her at the boundary, a heuristic generated the idea, an observation became a hunch, the time-box gave her room to chase it, and the notes turned it into evidence. That is exploratory testing working as designed.

What made that a session and not clicking around

Pull the example apart and the discipline is four moving parts. Take any one away and it degrades into the flailing skeptics imagine.

Charter — a mission with a reason. Without it you wander; with it you hunt.
Time-box — a fixed window, usually 60 to 120 minutes. It forces focus and makes the work countable.
Notes — the running log of ideas, bugs, concerns, questions, and uncovered areas. The deliverable, not an afterthought.
Debrief — a short close that turns a session into findings, decisions, and the next charter.

This is session-based test management, and its quiet superpower is that it makes an activity people dismiss as unmeasurable into something you can actually account for. A tester who ran four sessions today can hand you four charters, four logs, the bugs found, and the areas explicitly left uncovered. You can see coverage as a map of charters run and charters owed — not a false percentage, but an honest picture of where the team has looked and where it hasn't. Exploration stops being "we messed with it for a while" and becomes a unit of work with an input, an output, and a boundary.

Set the disciplined session next to its counterfeit and the line is sharp:

	Disciplined session	Clicking around
Direction	Charter names a target and a reason	No charter; wander wherever
Time	Fixed time-box, then debrief	Until bored or interrupted
Record	Running log; notes are the deliverable	Nothing written down
Ideas	Heuristics generate hypotheses	Same happy path every time
Output	Findings, bug reports, next charter	Vibes
Accountable	Charters run vs. charters owed	Unmeasurable

The counterfeit is what gives the practice its bad name. The discipline is what earns the twenty-minute P1.

A charter and session sheet you can actually run

Here is the observable version — what a good charter and a good session sheet look like, so the practice is accountable rather than vibes.

A charter that works has three parts: a target (the feature or risk area), a reason (the kind of bug you suspect lives there), and bounds (the time-box and the build). Compare:

Too vague: "Test the dashboard." No target you can aim, no reason, no end.
Too narrow: "Verify the dashboard loads for admin users." That's a scripted check wearing a charter's clothes — no room to discover.
Right: "Explore the dashboard with a test account that has 50+ workspaces, looking for performance and pagination breakdowns. 60 min, build RC-2.14." A place to hunt, a reason to hunt there, a boundary.

A useful move for generating charters is to point your acceptance criteria at their own edges: for every "given/when/then" the team agreed on, write a charter that hunts the unstated conditions around it — the interrupted "when," the malformed "given," the concurrent "then." The criteria tell you what was scoped; the charter goes looking for what wasn't.

A session sheet is one page per session with a fixed skeleton, so nothing gets lost and any teammate can read it cold:

Field	What goes in it
Charter	The one-sentence mission (target · reason · bounds)
Tester + build	Who ran it, against which build
Time-box	Planned duration and actual
Test ideas tried	What you attempted (even the dead ends)
Bugs	Findings with repro, reliability, conditions
Concerns / questions	Off feelings; things needing a real answer
Areas not covered	What you skipped and why — the honest gap
Debrief	What you learned; the next charter it suggests

The two rows people are tempted to drop are the two that make the sheet trustworthy. "Areas not covered" converts silence into a visible gap — the difference between "we didn't test that" being a known owed charter versus a surprise in production. And "debrief" is where a session earns compound interest: each one names the next charter, so the exploration deepens instead of circling the same happy path forever.

One more move closes the loop and keeps the scripts honest: every confirmed bug an exploratory session finds should become a scripted test. The workspace-leak race condition, once fixed, gets a regression check that switches mid-load and asserts no cross-tenant paint — so the ground exploration just took is now ground the scripts hold, and exploration is free to move to the next unmapped edge. That hand-off is the engine of the whole partnership: exploration discovers, scripts remember.

Run one charter this week

You don't adopt this by reading about it. You adopt it by running one session.

Pick the riskiest corner of something you shipped recently — the feature you'd least want a customer to find a bug in. Write one charter: a target, a reason, and a 60-minute box. Start a timer, keep a running log in the five buckets, and when the timer ends, spend five minutes on a debrief that names what you found, what you didn't reach, and the next charter it points to. Then take one confirmed finding and write it up as a real bug report.

One charter, sixty minutes, one page of notes. That is the entire discipline in a single sitting — and it is how a green build gives up the bug it was hiding. The team that keeps running charters, session after session, doesn't just find more bugs than the team that only scripts. It builds the one thing a script can never encode: testers who know where their system's bugs like to hide, and the discipline to go there on purpose.

ShiftQuality