Build a Test Data Factory — Hands-On Software Testing, Part 4

Shawn West
May 5
5 min read

Updated: Aug 24

Illustrative composite: Tolvern Freight's shipping suite, the system in Test Data Management Strategies. That article decides which isolation strategy a test gets. This builds the factory that makes the fast strategy practical.

Before you start

You need:

An ORM or data layer your tests can call directly.
An entity with at least two required relationships. A factory for a flat object is barely worth writing; the value appears with depth.
A test that currently sets up its data inline — you'll convert it in Step 4 and compare.
Ideally factory_boy (Python) or fishery/factory.ts (JS). Everything here works hand-rolled too.

About 60 minutes. Examples are Python with SQLAlchemy.

What you'll build

A factory that creates a shipment — and the customer, address, carrier account and rate card it requires — in one line, with values unique per test, letting each test name only the field it actually cares about.

Step 1: Find the repetition, and read what it hides (5 min)

def test_shipment_is_quoted():
    customer = Customer(name="Acme", plan="standard")
    session.add(customer); session.flush()
    origin = Address(customer_id=customer.id, country="GB", postcode="EC1A 1BB")
    dest = Address(customer_id=customer.id, country="FR", postcode="75001")
    account = CarrierAccount(customer_id=customer.id, carrier="DHL", account_ref="A-1")
    session.add_all([origin, dest, account]); session.flush()
    shipment = Shipment(customer_id=customer.id, origin_id=origin.id,
                        dest_id=dest.id, weight_kg=12)
    session.add(shipment); session.commit()

    assert quote(shipment).amount > 0

Eleven lines of setup for a one-line assertion. Worse than the volume: you cannot tell which of those values the test depends on. Is the assertion about weight_kg=12? About the GB→FR route? About DHL? A reader has to run it to find out.

Check: in your own equivalent test, try to name the fields the assertion actually depends on. If you're unsure about any of them, the setup is hiding the test's intent, and that's the problem the factory solves — not the typing.

Step 2: Write the base factory (15 min)

# tests/factories.py
import factory
from factory.alchemy import SQLAlchemyModelFactory

class CustomerFactory(SQLAlchemyModelFactory):
    class Meta:
        model = Customer
        sqlalchemy_session_persistence = "flush"

    name = factory.Sequence(lambda n: f"Customer {n}")
    plan = "standard"
    email = factory.Sequence(lambda n: f"cust{n}@tolvern.test")

Sequence is doing the load-bearing work. Every unique column must vary per instance, or the second test to run hits a constraint violation — and that surfaces as an order-dependent failure, not as a clear error.

Check: create two customers in one test and assert their emails differ. If they don't, every unique field in the factory is a future flake.

Step 3: Declare the relationships (15 min)

class AddressFactory(SQLAlchemyModelFactory):
    class Meta:
        model = Address
        sqlalchemy_session_persistence = "flush"

    customer = factory.SubFactory(CustomerFactory)
    country = "GB"
    postcode = factory.Sequence(lambda n: f"EC1A {n}BB")

class ShipmentFactory(SQLAlchemyModelFactory):
    class Meta:
        model = Shipment
        sqlalchemy_session_persistence = "flush"

    customer = factory.SubFactory(CustomerFactory)
    origin = factory.SubFactory(AddressFactory, customer=factory.SelfAttribute("..customer"))
    dest = factory.SubFactory(AddressFactory, customer=factory.SelfAttribute("..customer"),
                              country="FR", postcode="75001")
    weight_kg = 10

The SelfAttribute("..customer") is the line worth understanding. Without it, ShipmentFactory() creates three customers — one for the shipment and one for each address — and you get a shipment whose origin belongs to somebody else. The test still passes, because nothing checks. Then a permissions test fails six months later for reasons nobody can trace.

Check: create one shipment and assert s.origin.customer_id == s.customer_id. Run it before adding SelfAttribute and watch it fail. That failure is the reason this line exists.

Step 4: Convert the test and compare (10 min)

def test_shipment_is_quoted():
    shipment = ShipmentFactory(weight_kg=12)
    assert quote(shipment).amount > 0

Eleven lines to one. More usefully: weight_kg=12 is now visibly the thing the test is about, and everything unstated is visibly not the thing.

Check: delete the weight_kg=12 override and see whether the test still passes. If it does, the test never depended on the weight and the override was noise — remove it. If it fails, you've just documented the dependency in the only place that can't drift.

Step 5: Make persistence a decision (10 min)

Not every test needs a database row. Building one when you don't is pure cost.

ShipmentFactory.build(weight_kg=12)     # in memory, no INSERT
ShipmentFactory(weight_kg=12)           # flushed, has an id, visible to queries

Use build for anything testing pure logic — pricing, validation, serialisation. Use the persisting call when the code under test issues its own query.

Check: convert one pure-logic test to build and time the suite before and after. Small on one test; multiply by the number of pure-logic tests you have.

Step 6: The decision point — traits or new factories (10 min)

Eventually you need variations: an international shipment, an oversized one, a customer on a negotiated rate card. Two plausible routes, and the wrong one produces a factories file nobody can navigate.

The signal that decides it: does the variation change which fields exist, or only their values?

Values only → a trait on the existing factory.

class ShipmentFactory(SQLAlchemyModelFactory):
    ...
    class Params:
        oversized = factory.Trait(weight_kg=95, requires_pallet=True)

ShipmentFactory(oversized=True)

Different shape — extra required relationships, a different table, a different state machine → a separate factory. ReturnShipmentFactory is not a trait on ShipmentFactory; it needs an original shipment to exist.

Getting this backwards gives you either a factory with fifteen mutually exclusive boolean params, or twenty near-identical factories that drift apart.

Check: for each variation you're about to add, list the fields it changes. Only values → trait. Adds a relationship → new factory.

Step 7: Stop the factory from lying (5 min)

The failure mode nobody warns about: a factory that produces data the application could never create. Default status="delivered" on a shipment with no carrier booking, say. Tests pass against a state that cannot exist.

class ShipmentFactory(SQLAlchemyModelFactory):
    status = "pending"      # the state a real POST /shipments produces

Defaults should be the state the system produces at creation. Anything further along the lifecycle should be reached by calling the real transition, not by asserting it into existence:

shipment = ShipmentFactory()
book_carrier(shipment)          # the real path
assert shipment.status == "booked"

Check: for each default in your factory, ask whether the application could produce that value at creation. Any that couldn't is a test passing against an impossible state.

Step 8: Test the factory itself (5 min)

def test_factory_produces_a_valid_shipment(db_session):
    s = ShipmentFactory()
    assert s.id and s.origin.customer_id == s.customer_id
    assert s.status == "pending"
    assert ShipmentFactory().customer.email != s.customer.email

One test. It catches the three failures that otherwise appear as unrelated flakes elsewhere: broken graph, impossible default, non-unique values.

Check: it passes, and removing Sequence from the email field makes the last assertion fail.

The wrong factory beside the right one

	Inline setup	Over-general factory	This factory
Lines per test	~11	1	1
Intent visible	no	no — buried in params	yes, overrides only
Object graph consistent	by hand, often not	usually	asserted in Step 8
Unique values	manual	manual	Sequence
Params	—	15 booleans	traits + separate factories

You're done when

One line creates the full graph, and origin.customer_id == customer_id holds.
Two calls produce different values for every unique column.
Every override left in a test is one the test genuinely depends on (Step 4).
No default represents a state the application couldn't produce.
build is used wherever no database row is needed.

Troubleshooting

IntegrityError on unique constraint. A field that should be a Sequence is a constant. It will always fail on the second instance in a test.

The graph has duplicate parents. Missing SelfAttribute("..customer") — Step 3.

Factory data doesn't appear in the app's queries. Session mismatch: the factory and the app hold different sessions. This is the same wiring issue that breaks side-effect assertions in API testing.

Tests pass individually, fail together. Something is shared rather than created per test. Almost always a module-level factory instance or a leftover fixtures file.

The factories file is now 600 lines. Step 6 went the wrong way — traits used where separate factories were needed, or vice versa.

Factories make per-test data cheap, which is what makes transaction-based isolation practical — that decision is in Test Data Management Strategies.