top of page

Write Tests That Survive Refactoring — Test Automation in Practice, Part 3

  • Shawn West
  • Jul 8
  • 3 min read

Updated: Jul 28

Test Automation in Practice · Part 3

A test that breaks every time you refactor — even though no behavior actually changed — isn't protecting you; it's taxing you. Those tests are coupled to how the code works instead of what it does, and they slowly train a team to distrust the whole suite. This walks through rewriting tests to assert on behavior, so they stay green through refactors and only go red when something real breaks.

A test that breaks every time you refactor is worse than no test — it punishes good engineering. This tutorial walks through writing tests that fail when behavior changes and stay quiet when it doesn't.

The Core Principle

Test what the code does, not how it does it. Behavior, not implementation.

Step 1: Identify Implementation-Coupled Tests in Your Suite (15 min)

Walk recent failing tests. For each:

  • Did the failure indicate a real behavior change?

  • Or did the test fail because internal structure changed?

The latter are implementation-coupled.

Common signs:

  • Tests asserting on private method calls

  • Tests checking specific internal data structures

  • Tests asserting "called function X exactly N times"

  • Tests inspecting database internals (table structure, etc.)

Step 2: The "Refactor Test" (10 min)

For a test you suspect is implementation-coupled:

  1. Rename a variable in the code under test

  2. Run the test

If it fails, it was coupled to the variable name. Real test wouldn't care.

  1. Restructure the internal logic without changing behavior (e.g., extract a method)

  2. Run the test

If it fails, it was coupled to structure.

Now you know. Fix or accept.

Step 3: Refactor Tests to Behavior (30 min)

For an implementation-coupled test:

Before:

def test_create_user_calls_validator():
    validator = MagicMock()
    service = UserService(validator=validator)
    
    service.create_user("Sam", "sam@example.com")
    
    validator.validate_email.assert_called_once_with("sam@example.com")

This asserts on the internal call. If you refactor to call validation differently, the test breaks.

After:

def test_create_user_rejects_invalid_email(db_session):
    service = UserService(db_session)
    
    with pytest.raises(ValidationError):
        service.create_user("Sam", "not-an-email")

This asserts on the behavior: invalid email gets rejected. The internal call structure can change freely.

Step 4: Test the Boundary, Not the Internals (15 min)

Define the boundary of your unit:

  • For a class: the public methods

  • For a service: the public API

  • For a module: the exported functions

Test through that boundary. Don't reach inside.

If you need to verify a side effect, do so through observable state:

  • Database queryable

  • Event published (to a test subscriber)

  • File written (and readable)

  • Log entry (in a test logger)

These are observable from outside. Tests that verify them stay stable across implementation changes.

Step 5: Reduce Mocking (20 min)

Heavy mocking is the #1 source of implementation coupling.

Each mock is a thing your test asserts. Each assertion becomes a constraint on implementation.

Strategies to reduce:

  • Use real dependencies when they're fast (real DB via testcontainers)

  • Use fakes for slow dependencies (in-memory queue instead of real queue)

  • Use stubs (return values) instead of mocks (call assertions) when possible

  • Mock only what crosses an external boundary

# Heavy mocking — brittle
def test_create_user(mocker):
    mock_db = mocker.patch('app.db')
    mock_validator = mocker.patch('app.validator')
    mock_emailer = mocker.patch('app.emailer')
    
    service.create_user("Sam", "sam@example.com")
    
    mock_validator.validate.assert_called_once()
    mock_db.save.assert_called_once()
    mock_emailer.send.assert_called_once()

# Real dependencies — durable
def test_create_user(db_session, test_emailer):
    service = UserService(db=db_session, emailer=test_emailer)
    user = service.create_user("Sam", "sam@example.com")
    
    assert db_session.query(User).get(user.id).email == "sam@example.com"
    assert test_emailer.sent_to("sam@example.com")

Step 6: Avoid Asserting on Call Counts (5 min)

# Bad
mock.method.assert_called_once()

# Better — only when count is the contract
assert call_count == 1  # because of rate-limiting contract

"Called once" is rarely the actual requirement. Usually "produced the right outcome" is.

Step 7: Document Behavior in Test Names (10 min)

Test names are documentation. Use them.

# Bad
def test_function():
    ...

# Good
def test_create_user_succeeds_with_valid_email():
    ...

def test_create_user_rejects_invalid_email():
    ...

When a test fails, the name says what behavior broke. That's enormously valuable.

Step 8: Periodically Audit (ongoing)

Quarterly, review recently-changed tests. Why did they change?

  • Behavior changed (good — tests updated to match)

  • Implementation changed; tests had to track (bad — coupled)

  • Tests were just wrong (fix the test)

If most test changes track implementation, your tests are coupled. Invest in decoupling.

What You Just Did

You converted brittle tests into durable ones. The team can refactor freely; tests fail only when behavior actually changes.

Common Failure Modes

Mocking the system under test. Tests pass with mocks; production fails.

Over-asserting on internals. Mock.assert_called_once() everywhere. Tests resist refactoring.

Brittle string matching. Asserting on exact log messages that change in non-meaningful ways.

Snapshot tests of internals. Snapshots of internal state. Break on any refactor.

Continue the Test Automation in Practice path

Part of the Test Automation in Practice learning path.

bottom of page