top of page

Risk-Based Test Strategy Worksheet

Shawn West
1 day ago
4 min read

The dashboard said 87% coverage. The bug that reached customers lived in the 13%, in the one function that decided what they were charged.


The team had a coverage gate and a green build. Most of the tests exercised data models and helper functions, because those were easy to test. The discount logic, which combined three promotions and a loyalty tier, had two happy-path tests. A change to the order of operations shipped on a Friday, and for a weekend a subset of customers paid the wrong price.


(Composite example, assembled from patterns common in teams that manage to a coverage target. The figures are illustrative.)


Coverage tells you which lines ran. It doesn't tell you whether the tests would catch the failure that costs money. This worksheet starts from the other end: where does this system hurt most when it breaks, and is that where the tests are?


Step 1: map your risk


List the code paths where failure means lost money, lost data or lost trust. Test depth follows consequence, not how easy the code is to test.


Code path or feature

What failure costs

Risk (H/M/L)

Test depth

______

______

______

Deep

______

______

______

Deep

______

______

______

Light

______

______

______

Light


A common pattern, in our experience, is a suite that is heavy on utilities and data models and thin on the business rules that decide whether the product works. If your map shows that, invert it.


Step 2: test behaviour, not implementation


For each high-risk path, check that the tests assert on what the code does, not how it does it. A behaviour test survives refactoring; a test coupled to internals breaks on every refactor and, over time, makes the team afraid to refactor at all.


  • ☐ The test asserts on an observable result: a return value, a state change or a side effect.

  • ☐ The test doesn't reference private methods, private attributes or internal strategy.

  • ☐ The test would still pass if you rewrote the internals and kept the behaviour.

  • ☐ The test would fail if the output were wrong (the wrong price, the wrong total).


For example, calculator._apply_strategy(100) tests the internals and breaks on refactor; calculator.final_price(100) == 80.0 tests the behaviour the customer depends on.


Step 3: cover edge cases on the high-risk paths


A useful rule of thumb (ShiftQuality's, not a measured ratio): on high-risk paths, write about three edge-case tests for every happy-path test. For each high-risk behaviour, cover:


  • ☐ Empty or null input.

  • ☐ Boundary values: zero, negative, maximum, off-by-one, 29 February, quantity zero.

  • ☐ Concurrent or duplicate operations: a double form submit, simultaneous writes.

  • ☐ Dependency failures: an API returning 503, a database dropping mid-transaction, a timeout.


Step 4: contract-test the integration boundaries


Many expensive incidents happen where one component hands data to another, and unit tests can miss them by design: each side mocks the other, and both pass.


Boundary (A → B)

Contract to verify

Test written?

______

Shape, format and required fields match

☐

______

Both sides agree on null handling

☐

______

______

☐


Illustrative example: service A formats dates as MM/DD/YYYY and service B expects YYYY-MM-DD. Both pass their unit tests, and production quietly produces plausible but wrong dates. A narrow consumer-driven contract test catches this without a slow end-to-end suite.


Step 5: use coverage as a signal, not a target


  • ☐ Low coverage in a high-risk module: fix it, because that's a real gap.

  • ☐ The overall coverage percentage: don't treat it as a quality verdict.

  • ☐ On critical code, add mutation testing, which checks whether your tests would notice the code being wrong.


How to use it


  • Who: the QA or quality lead with one developer who knows the domain. Thirty to sixty minutes for a first pass on one service.

  • Output: a risk map with depth assigned, a list of behaviour tests to rewrite, the edge cases to add on high-risk paths, and the boundaries that need contract tests.

  • Done when: every row marked High has deep tests that assert behaviour, its edge cases are listed, and each boundary it touches has a contract test or a written reason it doesn't need one.


Where this changes


Very small codebases with a single owner can get away with less formality, because the risk map lives in one head. Regulated or safety-critical systems need more: a documented risk analysis tied to requirements and traceable evidence, not a one-page worksheet. And for AI features, where output varies run to run, apply the same risk-first thinking but with pass-rate thresholds; see risk-based testing for AI.


The reasoning behind this worksheet is in testing that catches real bugs. For contract tests in depth, read contract testing for microservices, and for checking your tests themselves, mutation testing. To write the result up as a strategy document, use the Test Strategy Template.


Final takeaway


The coverage number is for the report; the bug count is for the customer. Aim the suite at the places where failure costs something, test what those places do rather than how, and check the boundaries where data changes hands.


Sources


  • Google Testing Blog, Testing on the Toilet: Test Behavior, Not Implementation, August 2013. https://testing.googleblog.com/2013/08/testing-on-toilet-test-behavior-not.html

  • Martin Fowler, ContractTest. https://martinfowler.com/bliki/ContractTest.html

  • Pact documentation, consumer-driven contract testing. https://docs.pact.io/

  • PIT, mutation testing for the JVM, as an example mutation-testing tool. https://pitest.org/

bottom of page