top of page

Set Up Coverage Reporting — Test Automation in Practice, Part 5

S. West
Jul 8
3 min read

Updated: Oct 4

Test Automation in Practice · Part 5

Coverage is the most gamed number in testing: 90% can mean thoroughly tested or thoroughly meaningless, depending on whether those executed lines actually assert anything. Used well, though, a coverage report is a map of the code your tests never touch. This walks through setting up coverage reporting, wiring it into CI, and reading it for the real gaps — without sliding into worshiping the percentage.


Coverage tells you what code your tests executed. Not what they verified — those are different. This tutorial wires it up and walks through reading it correctly.


What You'll Build


Coverage tracking integrated into your test run, with reports visible in CI.


Step 1: Install Coverage Tooling (5 min)


Python:


pip install pytest-cov

Node:


vitest has coverage built in via --coverage.


npm install --save-dev @vitest/coverage-v8

Step 2: Configure (5 min)


Python — add to pytest.ini:


[pytest]
addopts = --cov=yourapp --cov-report=term-missing --cov-report=html

Vitest — in vitest.config.ts:


export default defineConfig({
  test: {
    coverage: {
      provider: 'v8',
      reporter: ['text', 'html', 'lcov'],
      include: ['src/**/*'],
      exclude: ['src/**/*.test.*', 'src/**/*.spec.*'],
    },
  },
});

Step 3: Run and Read (10 min)


pytest

or


npm test -- --coverage

Output includes per-file coverage percentages and total.


Open htmlcov/index.html (or coverage/index.html). Browse:


  • Files sorted by coverage

  • Click into a file to see line-by-line

  • Red lines: not exercised

  • Yellow: partially exercised (branches)


Step 4: Find the Real Gaps (15 min)


Click into low-coverage files. For each red line, ask:


  • Is this code that runs in production? → Should be tested.

  • Is this error handling for unreachable cases? → Document; consider removing.

  • Is this defensive code for impossible inputs? → Consider removing.


Don't add tests for every red line. Add tests for red lines that matter.


Step 5: Set Coverage Thresholds (10 min)


For critical code, fail builds below a threshold:


Python in pytest.ini:


[pytest]
addopts = --cov=yourapp --cov-fail-under=80

Vitest:


coverage: {
  thresholds: {
    statements: 80,
    branches: 75,
    functions: 80,
    lines: 80,
  },
}

Set thresholds at the level appropriate for your code. Critical modules higher; experimental lower.


Step 6: Add to CI (10 min)


In your GitHub Actions workflow:


- name: Run tests with coverage
  run: pytest --cov=yourapp --cov-report=xml

- name: Upload coverage to Codecov
  uses: codecov/codecov-action@v4
  with:
    file: ./coverage.xml

Codecov (or alternative) shows coverage trends, per-PR diffs, and posts comments on PRs.


Step 7: Configure Per-File Targets (10 min)


Different code deserves different coverage:


# codecov.yml
coverage:
  status:
    project:
      default:
        target: 80%
    patch:
      default:
        target: 90%  # New code coverage

component_management:
  individual_components:
    - component_id: critical
      paths:
        - src/auth/**
        - src/payments/**
      statuses:
        - type: project
          target: 95%

Critical paths get higher bars.


Step 8: Avoid the Coverage Worship Trap (5 min)


Things to remember:


  • 100% coverage with weak assertions tests nothing

  • 80% with strong assertions tests a lot

  • Coverage on unimportant code doesn't matter

  • Coverage on critical code matters a lot


Don't write tests just to hit numbers. Write tests for behaviors that matter.


Step 9: Branch Coverage > Line Coverage (5 min)


If your tool offers both, focus on branch coverage. A line with if x: can be 100% line-covered with one test (just enters the if). Branch coverage requires both true and false paths.


For most projects, configure branch coverage as the meaningful metric.


Step 10: Periodic Coverage Audit (ongoing)


Quarterly:


  • Walk the lowest-coverage files

  • Decide: needs coverage, or shouldn't be in the codebase?

  • Walk the highest-coverage files

  • Check: meaningful tests, or coverage-padding tests?


The combination keeps the suite honest.


What You Just Did


You have coverage tracking that surfaces gaps without becoming a target you game. The reports are diagnostic; the team responds to them deliberately.


Common Failure Modes


Coverage as target. "Must hit 90%." Tests written just to satisfy. Quality drops.


Vanity coverage. Tests that import a file and assert nothing. Coverage rises; tests verify nothing.


Ignoring branch coverage. Line-covered, branch-uncovered code. Bugs hide.


Coverage thresholds set arbitrarily. "We picked 80%." Match to risk: critical code higher; experimental lower.


Coverage reports nobody reads. Configured but never opened. Build it into the team's PR review.



Continue the Test Automation in Practice path



For the reasoning behind this step-by-step guide, read Test Coverage: The Honest Version.


Part of the Test Automation in Practice learning path.

bottom of page