Instrument Deploy Frequency — Engineering Metrics & DORA, Part 3

Engineering Metrics & DORA · Part 3

How often a team ships turns out to be one of the strongest signals of engineering health — not because shipping is the goal, but because frequent deploys are only possible when a lot of other things (small batches, automated tests, safe rollbacks) are already working. This walks through instrumenting deploy frequency from your CI/CD, benchmarking it against the DORA bands, and reading what a low number is really telling you.

Deploy more often, in smaller batches. Lower risk per deploy. Faster feedback.

Before you start

• Deploy events you can pull programmatically from CI/CD, with a timestamp and a target environment on each. If deploys happen partly by hand, decide now how those are recorded — an untracked manual release is a deploy that never happened, and it will flatter this number.

• A decision about what "production" means for you. Multi-tenant rollouts, dark launches and staged regional releases all need a rule, and the rule matters more than which one you pick.

• Service boundaries you can group by. An organisation-wide count averages a daily-deploying service with a quarterly one and describes neither.

• The context around the number, not just the number. Step 9 exists because deploy frequency is the metric most often used to rank teams who do not have comparable work.

Step 1: What Counts as a Deploy (10 min)

Define:

• Production environment only (or include staging?)

• New code (not config-only) — or both?

• Per-service or per-monorepo?

Pick a definition. Document. Apply consistently.

Step 2: Collect From CI / CD (10 min)

Most CI/CD systems log deploys:

• GitHub Actions: workflow run history

• ArgoCD / Spinnaker: deploy records

• Custom scripts: log each deploy event

For each deploy:

• Timestamp

• Service

• Commit SHA

• Team / owner (via tags)

Step 3: Per-Service vs. Aggregate (10 min)

Aggregate: "we deploy 50 times a day."

Per-service: "checkout deploys daily; analytics deploys weekly."

Per-service is more useful. Different services have different deploy needs.

Find services that deploy rarely. Why?

Step 4: Bench Against DORA Bands (5 min)

Per service:

• Multiple per day → elite

• Daily-ish → high

• Weekly to monthly → medium

• Quarterly → low

Most enterprises sit in medium/low. Goal: move up.

Step 5: Why Frequent Deploys Help (10 min)

• Smaller batches = smaller blast radius if buggy

• Faster feedback (find bugs while context fresh)

• Lower coordination overhead (don't pile changes)

• Easier rollback (less to undo)

• Team confidence ("we ship; nothing breaks")

Frequent deploys are an outcome of healthy practices.

Step 6: What Enables Frequent Deploys (15 min)

• Strong CI (catch issues before merge)

• Trunk-based development

• Feature flags (decouple deploy from release)

• Fast deploy automation

• Rollback automation

• Health checks / monitoring

Most teams can't go from monthly to daily by force. The supporting structure must come first.

Step 7: Common Blockers (10 min)

• Manual approvals — bottleneck on humans

• Long testing window — change accumulates

• Coordination across teams — wait for someone

• Database migrations — coupled to deploys

• Fragile production — fear of deploying

Each is fixable. Not all at once.

Step 8: Continuous Delivery (10 min)

Continuous delivery: every merged change goes to production automatically.

Requires:

• Trust in tests (no surprises)

• Trust in monitoring (catch issues fast)

• Feature flag discipline (control release separately)

Worth it for most modern teams.

Step 9: Don't Punish Teams With Low Frequency (10 min)

Some services deploy rarely because:

• They're stable; no changes needed

• They're risky (legal / compliance)

• Their architecture doesn't allow frequent deploys

Don't optimize their frequency in isolation. The underlying structure is the issue.

Step 10: Visualize (5 min)

Dashboard:

• Daily deploy count

• Per-service trend

• Time-of-day distribution

• Outage correlation (deploys vs. incidents)

Visible patterns:

• Friday deploy fear?

• 5pm push-and-pray?

Patterns surface anti-patterns.

What a low number is actually telling you

Deploy frequency is a symptom, and treating it as the disease is the classic misuse. Before setting a target, work out which of these you have — the remedy is different in each case, and only the last one is about deploying more.

What you observe	What it usually means	What to change
Infrequent, large deploys	Batching — work accumulates because releasing is expensive	The cost of releasing, not the schedule. Frequency follows automatically once a deploy is cheap
Frequent deploys, frequent failures	Releasing outpaced the safety net	Rollback, staged rollout and test feedback speed, before pushing frequency any higher
Regular but rigid cadence	A release process gated on something manual — a change board, a QA window, a shared environment	Find the gate. The cadence is a shadow of it, and the cadence will not move until it does
Genuinely low need	The service is stable, the domain moves slowly, and nobody is waiting on it	Nothing. A quarterly-deploying billing service is not underperforming, and Step 9 is the section to reread before someone puts it on a leaderboard

Benchmarking without the benchmark misleading you

Step 4 puts your number against DORA's published bands. Two cautions before you quote a tier anywhere.

Name the report year. DORA's thresholds have changed between annual reports — a team that reads as one tier against one year's table can land differently against another. A band without a year attached is unanchored, so cite the year and check the current report rather than a number remembered from a conference talk.

Bands describe clusters, not goals. The tiers came out of grouping surveyed teams; they were never a target to be hit. The useful question is not which band you are in, it is whether your own frequency is trending toward or away from the batch size you want — measured against your history, not someone else's cluster.

You're done when

• Deploy events land in your dataset automatically, with an environment tag, and you can state what share of your releases bypass that path.

• You can produce the number per service as well as in aggregate, and you can name the service whose frequency differs most from the average and say why that is legitimate.

• You can say which of the four situations above your lowest-frequency service is in — and if the answer is batching, you can name the specific cost that makes releasing expensive there.

What You Just Did

Deploy frequency: definition, data collection, per-service, DORA bands, why frequent helps, enablers, blockers, continuous delivery, no punishment, visualization. Frequency as signal.

Common Failure Modes

Optimize frequency without enablers. Higher failure rate.

Aggregate hides per-service issues. Some services stuck.

Punish low frequency. Doesn't fix root cause.

Force without trust. Engineers push and pray.

No correlation with incidents. Don't know if frequency helps.

---

Sources

• DORA, DORA's software delivery performance metrics — the current definition of deployment frequency and the wider metric set.

• DORA, Research — the State of DevOps report archive, 2014 onward. Band thresholds sit inside the individual annual reports and differ by year; quote the year alongside any tier.

Continue the Engineering Metrics & DORA path

• Previous — Part 2: Instrument Lead Time

• Next — Part 4: Instrument Change Failure Rate

Part of the Engineering Metrics & DORA learning path.