top of page

Form a Hypothesis from Symptoms — Debugging Systematically, Part 1

S. West
Jul 9
3 min read

Updated: Oct 4

Debugging Systematically · Part 1

The difference between an hour of debugging and a day of it is rarely intelligence — it's method. Poking at code and hoping is slow and lucky; forming a hypothesis and testing it is fast and repeatable. This walks through the scientific-method version of debugging: turn a vague symptom into a specific, testable guess, then design the cheapest experiment that proves it right or wrong.


Bad debugging: guess; change something; check if better. Good debugging: hypothesis, prediction, test.


Step 1: State the Symptom Precisely (5 min)


"The site is broken" isn't a symptom. Symptoms are observable:


  • "Users see 500 errors on POST /api/login starting at 14:32 UTC."

  • "The image upload times out after 30 seconds for files > 5 MB."

  • "Memory usage grows by 100 MB per hour without users."


Concrete; observable; measurable. The first 10 minutes of debugging are often clarifying what's actually happening.


Step 2: Gather Evidence (10 min)


Before forming a hypothesis:


  • What does the user see? Screenshot, repro steps.

  • What do logs say? Timestamps, error messages, stack traces.

  • What changed? Deploys, config, traffic, dependencies.

  • When does it happen? Always? Some users? Some times?


Evidence-first. Hypotheses without evidence are guesses.


Step 3: List Possible Causes (10 min)


Brainstorm:


500s on POST /api/login could be:
- Database overloaded
- Auth service down
- Bug in recent deploy
- Bad input from client
- Rate limit triggered
- Network issue between service and DB
- Wrong env var deployed

Cast a wide net. Order doesn't matter yet.


Step 4: Rank by Likelihood (10 min)


For each cause, ask:


  • Does the evidence support it?

  • Did anything change recently in this area?

  • How common is this failure mode?


Recent deploy 30 min before issue → strong signal
First 500 was at exact deploy time → very strong
Other services healthy → less likely DB / network

Rank highest-likelihood at top.


Step 5: Form a Specific Hypothesis (10 min)


A good hypothesis:


  • Specific ("the deploy at 14:32 introduced a bug in login validation")

  • Testable ("if I revert, errors should stop")

  • Predicts something ("then logs will show fewer 500s")


Bad hypothesis:


  • "Something with auth is broken." (Vague; not testable.)

  • "Probably a memory leak." (No evidence.)


Step 6: Predict What You'd See If True (10 min)


If the hypothesis is right:


  • "I should see error rate drop to baseline after rollback"

  • "git log shows changes to auth code in the deploy"

  • "I can reproduce by running the new code locally with the failing inputs"


If the hypothesis is wrong, predictions don't hold. Use this.


Step 7: Design a Cheap Test (10 min)


Each hypothesis is testable; some tests are expensive (rollback prod) and some are cheap (read git log).


Start with the cheapest tests:


  1. Read git log: 30 seconds

  2. Reproduce locally: 5 minutes

  3. Rollback canary: 10 minutes

  4. Full rollback: 20 minutes


If the cheap test rules it out, move on.


Step 8: Test (varies)


Run the test. Look at the result. Compare to your prediction.


If matches: hypothesis supported. Confirm with one more independent check before declaring victory.


If doesn't match: hypothesis ruled out. Update your ranking; pick the next.


Step 9: One Variable at a Time (10 min)


When testing, change one thing. If you change three and the issue resolves, you don't know which mattered.


In an outage, you might be tempted to throw five fixes. Do one, see if it works, then the next.


(Exception: when the cost of slow resolution is high, parallel mitigations may be worth it. But pause and clarify after.)


Step 10: Document the Process (5 min)


Even if you found it fast, write down:


  • The symptom

  • The hypotheses considered

  • The evidence for/against each

  • What ruled out what

  • What the root cause turned out to be


For the post-mortem (tutorial 9) and for next time. Patterns repeat.


What You Just Did


A repeatable debugging process: precise symptoms, evidence, ranked hypotheses, predictions, cheap tests. The scientific method applied to your code.


Common Failure Modes


Jump to a hypothesis without evidence. Often wrong; wastes time.


Vague symptoms. "Slow" instead of "p99 latency is 5s vs. baseline 200ms."


Change five things at once. Fix maybe works; never know which.


Trust the first plausible cause. Confirmation bias; missed actual cause.


No documentation. Same bug; future you doesn't remember.



Continue the Debugging Systematically path



Part of the Debugging Systematically learning path.

bottom of page