Tutorial 2: Set Up the Test Pyramid

Contributor
3 days ago
3 min read

The test pyramid (or trophy) is the deliberate allocation of testing effort across levels. Without it, teams accumulate tests randomly and end up with slow, fragile suites. This tutorial walks through implementing one.

What You'll Build

A test suite organized by level, with clear allocation and folder structure.

Step 1: Inventory What You Have (15 min)

Look at your existing tests:

find tests -name "*.test.*" -o -name "test_*.py" | head -30

For each, ask: what level is this? Unit, integration, or E2E?

You probably have:

A mix
No clear organization
Some heavily-mocked unit tests that are really integration tests in disguise
Slow tests labeled as unit tests

Step 2: Define Your Levels (10 min)

For your codebase:

- Unit: pure logic, no I/O, <100ms each
- Integration: with real database / real services internal to your system, <500ms each
- E2E: full stack via UI or public API, <30s each
- Smoke: subset of E2E for fast critical-path verification, <5 min total

Document these clearly.

Step 3: Reorganize the Folder Structure (30 min)

tests/
  unit/
    test_pricing.py
    test_validators.py
  integration/
    test_user_service.py
    test_order_service.py
  e2e/
    test_signin_flow.spec.ts
    test_checkout_flow.spec.ts
  smoke/
    test_smoke.spec.ts

Move existing tests to their correct level. Some will reveal themselves as miscategorized (e.g., "unit" tests that hit the database).

Step 4: Configure Per-Level Running (15 min)

In package.json or Makefile:

{
  "scripts": {
    "test:unit": "vitest run tests/unit",
    "test:integration": "vitest run tests/integration",
    "test:e2e": "playwright test",
    "test:smoke": "playwright test --grep @smoke",
    "test:all": "npm run test:unit && npm run test:integration && npm run test:e2e"
  }
}

Each level can run independently. Fast feedback for developers.

Step 5: Set Allocation Targets (10 min)

Aim for a shape that fits your context:

Classic pyramid:

70% unit, 20% integration, 10% E2E

Trophy (more common for orchestration-heavy services):

5% static, 25% unit, 60% integration, 10% E2E

These are rough targets. Don't optimize the count; optimize the value.

Step 6: CI Allocation (15 min)

Different levels run at different cadences in CI:

jobs:
  unit:
    runs-on: ubuntu-latest
    steps:
      - run: npm run test:unit
  
  integration:
    runs-on: ubuntu-latest
    services:
      postgres: { ... }
    steps:
      - run: npm run test:integration
  
  smoke-e2e:
    runs-on: ubuntu-latest
    steps:
      - run: npm run test:smoke

  full-e2e:
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
      - run: npm run test:e2e

Unit, integration, and smoke run on every PR. Full E2E runs on merge to main.

Step 7: Speed Targets (10 min)

| Level | Target | |-------|--------| | Unit (full) | <60s | | Integration (full) | <5min | | Smoke E2E | <5min | | Full E2E | <30min |

If you exceed targets, optimize. Slow tests get run less often.

Step 8: Diagnose Misallocation (ongoing)

Watch for:

Tests labeled "unit" that take seconds — really integration tests
Tests labeled "integration" that mock everything — really unit tests
Tests labeled "E2E" that don't go through the UI — really integration tests

The label should match the actual level.

What You Just Did

You converted ad-hoc test accumulation into deliberate allocation. The team now has clear categories, fast feedback per category, and a shape they're targeting.

Common Failure Modes

Misclassified tests. Slow unit tests; mocked integration tests. The labels lie.

Static targets. Allocation set once; never adjusted. As the system evolves, the right allocation changes.

Over-emphasis on E2E. Heavy investment in E2E because they "test everything." Result: slow, brittle suites.

Under-emphasis on integration. Skipping the level that catches most real bugs.

Manual running. Tests exist but aren't wired to CI by level.

ShiftQuality