Smoke Testing: The Five-Minute Quality Gate

Contributor
Mar 13
5 min read

A smoke test is a quick check that the basic flow of a system works. Named for the electrician's practice of plugging in new wiring and watching for smoke — if it doesn't catch fire immediately, you can do the real testing. Smoke tests catch the catastrophic-but-obvious issues before you waste time on deeper testing.

This guide is the practical version for software teams.

What Smoke Tests Verify

The minimum the application has to do. Not all features, not all edge cases, not all platforms — the core, end-to-end functionality.

For a typical web application:

The app loads (HTML returns 200)
A user can sign in
The main page loads with the user's data
The primary action can be completed
The user can sign out

That might be five tests. Total runtime: under five minutes.

If smoke passes, the application is fundamentally working. Deeper testing is worthwhile.

If smoke fails, something fundamental is broken. Stop and fix that before doing other testing.

When to Run Smoke

After every deploy to any environment
Before any meaningful manual or automated test run
As the first stage of any release pipeline
As a health check during incident response

Smoke runs are cheap. Run them often.

Smoke vs. Sanity vs. Health

These terms get used interchangeably; they're slightly different.

Smoke: verifies basic end-to-end functionality after a build/deploy
Sanity: verifies a specific area is working (more focused than smoke)
Health check: continuous monitoring, usually a single endpoint

You can have all three. They serve different purposes.

The Smoke Test Suite Shape

A working smoke suite has 5-20 tests covering:

Application starts and responds
Authentication works
Primary user journey completes (one path)
Key integrations are reachable
Critical pages render

What's not in smoke:

Edge cases
All user roles
All platforms or browsers
Validation rules
Error paths

Those belong in the broader test suite, not the smoke check.

Speed Is the Point

A smoke suite that takes 30 minutes is not a smoke suite. The whole reason smoke exists is to fail fast.

Target: under 5 minutes for the full smoke suite. Under 2 minutes is better.

Speed techniques:

Run smoke tests in parallel
Skip setup that isn't needed for the smoke flow
Use API-level shortcuts where possible (set up state via API, verify via UI)
Avoid third-party dependencies in smoke if you can

If smoke is slow, the rest of the pipeline waits for it. Treat smoke speed as a first-class concern.

What Smoke Doesn't Catch

Smoke is fast and shallow. It misses:

Edge cases in business logic
Performance regressions
Permission boundary issues
Subtle UI bugs
Issues affecting specific user types

That's fine. Smoke isn't supposed to catch these. Deeper test layers catch them. Smoke's job is "is the system fundamentally working?"

Smoke for Microservices

In a microservices architecture, each service has its own smoke. The composite smoke verifies cross-service flows.

Each service: starts, health check passes, can talk to its database
Cross-service: the primary user flow works through the full stack
System: critical integrations between services succeed

Each level can run independently or as part of a larger pipeline.

Smoke in Production

Production smoke tests serve a different purpose: continuous verification that the system is healthy.

A production smoke might:

Run every 5 minutes
Execute a synthetic transaction
Check key metrics are within bounds
Alert if anything fails

Synthetic monitoring services (Datadog, Pingdom, etc.) often provide this. It's smoke testing as a continuous health check.

Building a Smoke Suite

If you have nothing today:

List the top 3-5 user actions in your application
For each, write a test that executes the happy path end-to-end
Add an "application is alive" check (load the homepage; HTTP 200)
Total runtime should be under 5 minutes
Run smoke before any other test phase

That's the entire investment to get a working smoke suite.

Smoke as a Diagnostic

When a deploy fails smoke, the failure is diagnostic.

Application doesn't start → infrastructure or build issue
Login fails → auth service or session issue
Primary action fails → backend or database issue
Page doesn't render → frontend or asset delivery issue

Smoke failures point at the broad area of the problem before deeper investigation begins.

What Goes Wrong

Smoke that's not actually smoke. A 200-test suite called smoke that takes 20 minutes. Defeats the purpose.

Smoke that flakes. Loses trust. The team starts ignoring smoke failures. Fix flakiness as a priority.

Smoke that's not run. Suite exists but is bypassed in some deploy paths. Defeats the purpose.

Smoke that's too shallow. Two tests, neither exercises real flow. Misses obvious breakage.

Tying Smoke to Deploys

A working pattern: smoke runs as part of every deploy, and the deploy is rolled back automatically if smoke fails.

Deploy → Run smoke → Pass: continue. Fail: roll back.

This makes smoke a quality gate, not just a check. Failed smoke produces immediate corrective action.

For canary or blue-green deployments, smoke runs against the new version before traffic shifts. Same principle.

A Worked Example

Smoke suite for an e-commerce site:

1. GET / returns 200 with expected HTML structure
2. POST /api/auth/login with test credentials returns a token
3. GET /api/user/me with the token returns the test user
4. GET / with the token shows authenticated UI elements
5. GET /api/products returns at least 1 product
6. POST /api/cart with a product adds it
7. GET /api/cart returns the added item
8. POST /api/checkout in test mode completes

Eight tests. Each verifies one specific thing. Total runtime ~90 seconds. If all pass, the site is fundamentally working.

Maintaining Smoke

Smoke tests need ongoing attention:

When a primary flow changes, smoke updates first
When a smoke test breaks, fix immediately (or remove if obsolete)
Periodically review: is the smoke still covering what matters?

Stale smoke tests are silent. They pass while real failures happen. Worse than no smoke.

Anti-Patterns

Smoke as e2e. Comprehensive UI tests labeled smoke. Too slow, defeats purpose.

Smoke as health check. A single ping. Too shallow; doesn't verify functionality.

Smoke without rollback. Suite runs, fails, deploy continues. The fail produces nothing.

Smoke ignored. "Smoke is flaky; we run it but ignore failures." Defeats purpose.

Key Takeaway

Smoke tests verify the basic end-to-end function of a system, fast. 5-20 tests, under 5 minutes total. Run after every deploy, before every test phase. Tie to deploy rollback so failures produce action. Keep smoke focused — it doesn't replace deeper testing, it gates whether deeper testing is worthwhile. Maintain smoke as primary flows evolve. Five minutes of smoke prevents hours of wasted time downstream.

ShiftQuality