API Testing: A Practical Walkthrough

An API test that asserts on the response is asserting on the cheapest thing the endpoint produces. The walkthrough that matters isn't which library to use — it's deciding, per endpoint, which of the four surfaces you're actually willing to leave unwatched.

Tolvern Freight ran 340 API tests against its shipping service. Green for eight months. Then a customer called: their shipment had a tracking number in Tolvern's dashboard and no record at the carrier.

The endpoint had been returning 201 with a perfectly well-formed body the whole time. On one branch — customers with a negotiated rate card — the carrier booking was never written. The response didn't know that. Neither did any of the 340 tests, because every one of them made a request and read the reply.

The response is one surface out of four

An HTTP call to a real service produces more than a reply. It produces four things you can assert on, and they fail independently:

1. The response — status code and body.

2. The persisted effect — what the system now owns that it didn't before.

3. The next read — what a subsequent GET returns, which is not the same object the POST handed back.

4. The concurrent view — what a simultaneous caller sees mid-flight.

Tolvern's bug lived entirely in surface 2. No amount of care on surface 1 would ever have reached it, because the code path that wrote the carrier booking ran after the response body was assembled.

This is the whole mechanism, and it's less about testing than about where the truth lives. The response is a claim the service makes about itself. The other three surfaces are the service being checked against reality. A suite built only on claims will be green exactly as long as the service's self-description stays accurate — which is to say, until the interesting bug.

The test you can run: open the last API test you wrote. Count the assertions that touch anything other than the response object. If the answer is zero, you have a suite of claims.

What each surface costs, and what only it can catch

They are not equally worth buying. Read this as a spending decision, not a checklist.

Surface	The failure class only it catches	Typical cost	The assertion that proves it
Response	Wrong status, wrong shape, leaked fields, bad error body	~0 — you already have the object	assert r.status_code == 201 and a shape check
Persisted effect	Work the response claimed but never did	One extra query or one downstream stub	Read the row or assert the outbound call happened
Next read	Fields the writer computes and the reader recomputes differently	One extra request	GET the resource and compare the fields that matter
Concurrent view	Lost updates, duplicate creates, broken invariants under load	High — slow, and the most common source of flakes	Fire N in parallel; assert the invariant, never the timing

Notice the third row. The POST response and the subsequent GET are produced by different code, often by different serializers, sometimes by different services. When they drift, every response-only test stays green and every client breaks. Tolvern found a second bug this way within a week: POST /shipments returned estimated_delivery as a date, GET /shipments/{id} returned it as a full timestamp, and the mobile client had been parsing whichever it got.

The test: for one endpoint that creates something, GET it back and diff the two bodies field by field. Every field that differs is either a deliberate decision nobody wrote down or a bug.

Read-back is the cheapest surface and the one most often skipped

Surface 2 and surface 3 cost one extra line each. That's the entire price.

def test_create_shipment_books_the_carrier():
    r = client.post("/shipments", json=valid_shipment, auth=rate_card_customer)
    assert r.status_code == 201
    shipment_id = r.json()["id"]

    # Surface 2: the work the response implied
    booking = db.query(CarrierBooking).filter_by(shipment_id=shipment_id).one_or_none()
    assert booking is not None, "201 returned but no carrier booking was written"

    # Surface 3: what the next caller actually sees
    got = client.get(f"/shipments/{shipment_id}", auth=rate_card_customer)
    assert got.json()["estimated_delivery"] == r.json()["estimated_delivery"]

Two assertions. Both of Tolvern's eight-month bugs die here.

Note the fixture: rate_card_customer, not a generic user. The broken branch was the negotiated-rate branch, and a suite that only ever exercises the default customer never enters it. The surfaces tell you what to assert; your fixtures decide which code paths you assert it on. Getting the first right and the second wrong just moves the blind spot.

The test: list the branches in your endpoint that depend on the caller rather than the payload — plan tier, feature flag, region, permissions. Count how many have a fixture. That count is your real coverage of that endpoint, whatever the line number says.

Where the surfaces are supposed to disagree

Asserting that read-back equals the write response is wrong the moment anything is asynchronous, and applying the rule mechanically produces a suite that fails for correct behaviour — which is worse than not having it, because teams learn to rerun it.

Three cases where disagreement is the design:

• Eventual consistency. The POST returns status: "pending"; the GET a second later returns "booked". Asserting equality here is asserting the system doesn't work. Assert the transition is legal instead, or poll to a terminal state with a timeout.

• Server-computed fields. updated_at, sequence numbers, ETags. Exclude them explicitly and by name — never with a loose "ignore extra fields", which also swallows the field you meant to catch.

• Deliberate projection differences. The list view returns a summary, the detail view the full object. Fine — but this should be written down somewhere, and read-back tests are how you find out it isn't.

The distinction worth holding: you are asserting a relationship between the surfaces, not their equality. Sometimes the relationship is "identical", sometimes "eventually identical", sometimes "a strict subset". Naming which one applies is the actual design work.

The test: for each field your read-back test excludes, say out loud why. Any field where the answer is "because the test failed and this made it pass" is a bug you have already found and are now suppressing.

The one surface worth skipping most of the time

Surface 4 is real and expensive. Concurrency tests are slow, they need genuine parallelism rather than a threading approximation, and they are the single largest contributor to flaky suites — because a test that fires ten requests and asserts on what happened when is asserting on the scheduler.

Buy it only where a lost update actually costs something: money, inventory, idempotency keys, anything with a uniqueness constraint the database isn't already enforcing for you. For the rest, a unique index is a cheaper and stronger guarantee than a test.

When you do write one, assert the invariant, never the sequence:

def test_concurrent_creates_dont_double_book():
    results = run_in_parallel(lambda: client.post("/shipments", json=same_idempotency_key), n=10)
    assert sum(r.status_code == 201 for r in results) == 1
    assert db.query(CarrierBooking).filter_by(shipment_id=...).count() == 1

"Exactly one succeeded" holds under any interleaving. "The second one got a 409" holds only under the interleaving you happened to observe on your laptop.

The test: for every concurrency test in your suite, ask whether its assertion would still be true if the operating system scheduled the threads in the reverse order. If not, you have written a flake and scheduled its first failure for a busy CI day.

What to change in your next test

Take one endpoint — the one that would embarrass you most if it silently did nothing. Add two lines: read the side effect, then read the resource back. Run it against the branch that only fires for your least-default customer.

If both pass, you've spent four minutes confirming the endpoint is honest. If either fails, you have found the Tolvern bug in your own system, and it has been green the entire time.

Then decide, deliberately and per endpoint, which surfaces you're buying. Most endpoints justify one and two. Read-back earns its keep wherever a writer and a reader compute the same field. Concurrency is worth it where a duplicate costs money — and almost nowhere else.

The step-by-step build of exactly this test, one surface at a time, is in the companion tutorial: Test an API End-to-End. And when what you need to pin down is the agreement between two services rather than the honesty of one, that's a different instrument — contract testing.

Tolvern's suite is 380 tests now, up from 340. The forty that were added are the read-backs. They are the only ones that have caught anything since.