Test Coverage: The Honest Version

The badge on the repo said 85%. The team was proud of it — they had earned it over two quarters, and it gated every merge. And yet the same class of bug kept reaching production: an off-by-one on a threshold, a total that came back subtly wrong, a discount that fired one order too early. Each time, the postmortem ended the same way. Someone would pull up the coverage report, point at the green, and say, "But that file is covered."

So one afternoon we opened the covered file. The function had a test. The test called the function with one input and asserted expect(result).toBeTruthy(). That was the whole check. The line showed green because the code ran — not because anything confirmed it ran correctly. The coverage report was telling the truth. We were reading it wrong.

That is the honest version of coverage: it tells you what code your tests executed, not what they verified. Everything painful about coverage as a metric flows from confusing those two words. This piece is about keeping them apart — and about turning a coverage report back into the thing it is actually good at: a map of code nobody has tested yet.

What the number actually counts

Line, branch, function, statement — the coverage families differ in what unit they count, but they all answer one question: did this code run during the test suite? None of them looks at your assertions. None of them knows whether the test checked the output, checked the wrong thing, or checked nothing at all.

This is the mechanism the 85% badge hides. A coverage tool instruments your code, runs your tests, and records which instrumented points were hit. A function that gets called and never inspected counts exactly the same as a function whose every output is pinned down by a careful assertion. Execution is observable to the tool. Verification is not. The instrument was never built to see it.

Which means a high number is consistent with two very different worlds: a suite that thoroughly checks behavior, and a suite that merely walks through the code with its eyes closed. The number cannot tell you which world you are in. Only reading the tests can.

Runnable takeaway: pick your three most business-critical files and open their tests, not their coverage percentages. For each test, find the expect/assert line and ask what would have to break for it to fail. If the answer is "almost nothing," the green is decorative.

One function, 100% covered, still broken

Here is the trap in miniature. Take a real bulk-discount rule — the kind of money-touching logic that shows up in every commerce codebase.

(Developed example — composite scenario.)

// discount.js
function applyBulkDiscount(order) {
  const rate = order.quantity > 100 ? 0.10 : 0;   // 10% off for bulk orders
  return order.total * (1 - rate);
}

Now the test a team wrote to clear the coverage gate:

// discount.test.js
test('applies bulk discount', () => {
  const result = applyBulkDiscount({ quantity: 150, total: 1000 });
  expect(result).toBeTruthy();   // 900 is truthy. So is 1000. So is -5.
});

Run coverage on this and the report is spotless for the file:

File          | % Stmts | % Lines | % Branch | Uncovered
--------------|---------|---------|----------|----------
discount.js   |   100   |   100   |    50    |

Every statement ran. Every line is green. The one input, quantity: 150, executed the rate calculation and the return. By the line-coverage scoreboard, this function is done. But look at what the test actually promised: that the result is truthy. 900 is truthy. So is the un-discounted 1000. So is a buggy -5. The assertion cannot tell a correct discount from no discount from a negative charge. It verifies nothing about the behavior anyone cares about.

You do not have to take that on faith. Probe it the way a mutation tester would — introduce one small, deliberate defect and see whether the suite notices. Flip the comparison from > to >=:

const rate = order.quantity >= 100 ? 0.10 : 0;   // the mutant

That single character changes real behavior: an order of exactly 100 units now gets the discount when the rule says it shouldn't. Rerun the test. It still passes. The mutant survives — coverage 100%, mutation score 0 out of 1. The "covered" test sails straight past a genuine boundary bug, because it never tested the boundary and never checked the amount. This is the whole argument, reproduced on your own machine in five lines: coverage is not verification, and a green file can be a broken file.

Now the honest test — the one that earns the green:

test('discounts orders over 100 units, and not at the boundary', () => {
  expect(applyBulkDiscount({ quantity: 101, total: 1000 })).toBe(900);   // over → discounted
  expect(applyBulkDiscount({ quantity: 100, total: 1000 })).toBe(1000);  // at → NOT discounted
  expect(applyBulkDiscount({ quantity: 50,  total: 1000 })).toBe(1000);  // under → full price
});

This version kills the mutant. At quantity: 100 it demands 1000; the >= defect returns 900 and the test goes red. Same line coverage as before — 100% — but now the number means something, because the assertions pin the behavior at the exact place it can go wrong. The lesson is not "write more tests." It is that two suites with identical coverage can differ completely in what they guarantee. For how to write assertions that actually pin behavior, see Unit Testing Best Practices; for generating the boundary and beyond-boundary inputs automatically instead of hand-picking them, see Property-Based Testing: A Beginner's Tour.

Runnable takeaway: take one critical function this week and hand-run a single mutation — flip a comparison, swap a + for a -, delete a line. If the suite stays green, you have found a test that executes without verifying. Fix that one before you chase another percentage point.

The discovery move: ask what the test asserted

The failure in that story was not a testing failure. It was a discovery failure — nobody had asked the one question that separates execution from verification: what did this test assert?

That question reframes the whole report. A coverage tool draws you a map of your codebase and shades every line your tests touched. Read as a grade, the map invites you to inflate the shaded area until the average looks respectable. Read as a map, it does something far more useful: the unshaded lines are the code no test has ever run — the honest inventory of what you have not checked at all. That inventory is the legitimate product of a coverage run. The average is a byproduct.

So the move on every coverage report is two questions, in order. First, on the red: what is this untested code, and does it matter? Uncovered payment logic is a finding; an uncovered debug logger is noise. Second, on the green: what did the covering tests actually assert? Green is a claim of execution; it is your job to confirm it is also a claim of verification. The first question uses coverage for exactly what it measures. The second question refuses to let coverage answer a question it cannot see.

Runnable takeaway: in your next coverage review, sort by uncovered lines and read the top of that list as a to-do list of untested behavior. Then spot-check five green files and, for each, name the assertion that would catch a real regression. Two questions, every report.

Where coverage earns its keep — and where it lies to you

Coverage is a genuinely good diagnostic instrument and a genuinely bad target. The same number that is informative when you read it becomes corrupting the moment you chase it — the familiar shape of Goodhart's law, where a measure used as a goal stops measuring what you wanted. Keep the uses and the misuses side by side:

Coverage used as a diagnostic (good)	Coverage used as a target (bad)
Finding untested code — the red lines are an inventory of behavior no test runs	Chasing a percentage — writing assertion-free tests to move the average up
Sanity-checking a new PR — did the new logic get any tests at all?	Gating merges on a flat number — 85% everywhere, regardless of what the code does
Spotting dead code — lines no test can reach may be unreachable in production too	Rewarding the metric — teams optimize the number, not the risk it was meant to reduce
Watching the trend — a sudden drop flags a feature that shipped without tests	Trusting the badge — "it's covered" ends the conversation instead of starting it
Locating gaps by risk — is the critical code covered, not just the average?	Averaging across risk — glue code padding the number while payment logic stays thin

The left column treats the number as a pointer to somewhere worth looking. The right column treats the number as the destination. A gate built on the right column is theater; it passes the assertion-free discount test and blocks nothing that matters. If you want a merge gate that actually gates, it has to check for the thing coverage can't see — see Quality Gates That Actually Gate.

Runnable takeaway: audit your own coverage gate against this table. If it enforces a single flat percentage across the repo, you are on the right-hand side. Replace "the whole repo must hit N%" with "these specific critical paths must have asserting tests," and let the global number stay a diagnostic.

Setting targets by risk, not by decree

A flat target is where good intentions go wrong. Demand 90% everywhere and you over-invest in code that barely matters while under-investing where a defect is expensive — and you manufacture pressure to write exactly the empty tests that started this article. Coverage targets should track the cost of being wrong, not a company-wide round number.

A workable division, with the reasoning attached: critical code — payments, security, auth, anything that touches money or data integrity — earns a high bar, in the 90%-plus range, and every one of those tests must carry real assertions, because a survived mutant here is an incident. Standard business logic sits lower, around 70–85%, covered where behavior is non-obvious and skipped where it is trivial. Glue and boilerplate — generated code, framework wiring, dumb getters, UI scaffolding — is low priority; testing it inflates the number without reducing risk. Experimental or throwaway code may legitimately sit near 0% until it proves it will live. The numbers here are illustrative team targets, not research findings — the point is the shape (high where a bug is costly, low where it is cheap), not the specific figures. For how these tiers fit a layered suite from unit tests up to production canaries, see Testing Strategies That Scale.

Runnable takeaway: classify each top-level module as critical, standard, glue, or experimental, and write the target next to it. When a module's actual coverage and its risk tier disagree — glue at 95%, critical at 60% — you have found where your testing effort is pointed at the wrong place. Move it.

Line versus branch — the cheapest honesty upgrade

Look again at the report from the example: 100% lines, 50% branches. That gap is not a rounding artifact — it is the most common way line coverage flatters you. Line coverage counts whether a line ran; branch coverage counts whether each direction of a decision ran. An if (x > 100) with a single-line body can show 100% line coverage from one input that takes the true path, while the false path — the else that never executed — is a decision your tests have never made. In the discount function, the : 0 branch (no discount) was never taken, and that untaken branch is exactly where the boundary bug lived.

Line coverage is the most reported and the least useful of the families for this reason. Branch coverage is a strictly better signal for the same run, and turning it on usually costs one flag in your coverage config.

Runnable takeaway: enable branch coverage today and re-read your critical files. Every branch you are missing is a decision your suite has never forced the code to make — start your next tests there, at the paths the line number quietly skipped.

What to do Monday

You do not need a coverage overhaul. You need to stop reading the number as a grade. In one review cycle:

1. Flip the report from scoreboard to map. Sort by uncovered lines; read the red as untested behavior and triage it by risk, not by volume.

2. Spot-check the green. Open the tests for your five most critical files and name, for each, the assertion that catches a real regression. Empty assertions get fixed first.

3. Run one mutation by hand. Flip a comparison or delete a line in a critical function. If the suite stays green, you have a verification gap the coverage number can't show.

4. Turn on branch coverage. Same run, more honest signal; start new tests on the branches you were missing.

5. Replace the flat gate with risk tiers. Require asserting tests on critical paths; let the global percentage be a diagnostic you watch, not a target you chase.

None of this raises the badge. All of it makes the badge mean something — which is the only version of coverage worth having.

Sources

• Developed example (the applyBulkDiscount function, the assertion-free test, the >-to->= mutation, and the coverage figures) is an original composite scenario written for this article. The code and numbers are illustrative and reproducible, not drawn from a specific production incident. The 80% and 85% targets are illustrative team goals, not research findings.

• The distinction between a measure and a target reflects the standard formulation of Goodhart's law ("when a measure becomes a target, it ceases to be a good measure").

• The idea of probing a passing test by introducing a deliberate defect is the core premise of mutation testing, a well-established technique in the software-testing literature.

Keep learning. This article is part of the Software Testing Foundations path in the ShiftQuality Learning Center.