Shipping AI

Shipping AI isn't the last step of building it. It's a separate discipline, and most teams find that out after the launch.

Penhale Mutual's claims-triage assistant passed every demo. In the steering committee it summarised a messy first-notice-of-loss email in four seconds, flagged the likely fraud indicator and routed the claim. Everyone agreed it was ready. Eleven days after launch, the model provider had a partial outage during the Monday peak. The assistant didn't fail loudly. It hung. Adjusters saw a spinner in the panel they'd been trained to read first, and a spinner reads as slow, not broken. The queue backed up for fifty minutes before anyone escalated.

(Penhale Mutual is a composite insurer we use across ShiftQuality's AI-quality series, built from patterns we've seen. It isn't a real company, and its figures are illustrative.)

Nothing in the demo was false. The demo just answered a different question. It showed that the feature can work. Shipping needs to show what happens when it doesn't: when the model is down, when the output is plausible and wrong, when a prompt change quietly makes Tuesday's answers worse than Monday's. Those questions don't come up in a demo because nobody scoped them, and you can't test your way to a behaviour you never defined.

This guide covers that discipline. It's the ShiftQuality map for shipping AI to production: defining done, evaluating output you can't judge by eye, gating releases, rolling out safely, and running AI once real users depend on it. Each section links to the deep-dive articles and hands-on tutorials behind it.

Why AI features stall between pilot and production

Most AI pilots that don't reach production weren't let down by the model. They were let down by everything around it. A pilot runs on a curated dataset, a forgiving audience and a team that watches every output. Production removes all three at once. The data drifts, users find inputs nobody imagined, and nobody is watching outputs by hand.

The pattern we see most often is organisational, not technical. Nobody owns the eval set. Nobody can say who is allowed to override a failing gate. Nobody wrote down what the feature should do when the model is unavailable. Those aren't modelling problems, so a better model won't fix them.

• The Pilot-to-Production Gap Is an Org Problem, Not a Model Problem

• Pilot Theater: How to Tell Real AI Adoption from Demo Theatre

• Why a 95%-Reliable Model Builds a 60%-Reliable System. Chain ten steps that each succeed 95% of the time and the whole chain succeeds about 60% of the time (0.95¹⁰ ≈ 0.60).

Test you can run: for your current AI pilot, name the person who owns the eval set and the person who can approve shipping past a failed check. If either answer is "the team", you've found your first production blocker.

Define done before you build

An AI feature's acceptance criteria usually describe the feature working. The items that decide whether it survives production describe it failing: what the user sees when the model is down, how you roll back without a deploy, what confidence threshold sends a case to a human. Those items can't be shown in a sprint review, so they're the ones that get skipped. In Penhale's composite, across nine releases the two Definition of Done items that couldn't be demoed were met twice each.

So start shipping in discovery. Write requirements that name the failure behaviour, the data the feature depends on and how quality will be measured, before anyone writes a prompt.

• Writing Requirements for AI/ML Features

• Definition of Done for an AI Feature

• Quality Engineering for AI. The overall approach.

Test you can run: open your AI feature's Definition of Done and mark every item that can't be shown in a demo. Then check how many of those were actually met in the last release.

Evaluate what you can't eyeball

You can't QA a non-deterministic feature by clicking around. Say you change one word in a prompt and it looks better on the example you checked. Without a fixed test set you have no way to know whether it got worse on the forty you didn't check. Evals turn "it feels better" into a number you can defend. A good test strategy also places evals inside a wider set of layers: deterministic checks, contract tests on structured output, and human review where the risk justifies it.

• A Test Strategy for LLM Features (Not Just Evals)

• Testing Non-Deterministic Systems Without Losing Your Mind

• LLM Evaluation Frameworks: Measuring What You Cannot See

• Risk-Based Testing for AI: Where to Spend Your QA Budget

• Hands-on: Build Your First Eval Set · Evaluate Agent Reliability

Test you can run: take your last three prompt changes. For each one, can you produce a before-and-after score on the same fixed set of cases? If not, those changes shipped on judgement alone.

Gate the release, and treat overrides as data

A quality gate is the one part of the pipeline whose job is to refuse a release. That makes it the only place the standard becomes binding, and also the place people argue with most. In Penhale's composite, the eval gate blocked 31 of 88 releases in its first quarter, and 26 of those were overridden. The config said the threshold was 4.0. In practice it was "4.0, unless someone asks."

The fix isn't a stricter number. Make overrides visible, give them a named owner and review them as a set. The override rate tells you what the gate actually enforces.

• Quality Gates for Shipping AI

• Pilot-to-Production Readiness Checklist for AI Features

• Hands-on: Build an Eval CI Pipeline. Wire evals into CI so a regression fails the build.

Test you can run: count last quarter's blocked AI releases and how many were overridden. If more than half went out anyway, the gate is acting as a warning, not a control. Decide which of those it should be.

Roll out like the model will surprise you

Rolling out AI uses the same tools as any other risky release: feature flags, staged rollout and A/B comparison. One thing is different. The failure is often plausible output, not an error, so your rollback trigger can't depend only on error rates. Decide before launch which quality signal will make you roll back, and test the rollback path before you need it. Cost and latency belong in the rollout plan too. A feature that's correct but takes 12 seconds, or costs more per request than it earns, hasn't shipped successfully.

• Running LLMs in Production: Cost, Latency, and the Tradeoffs Nobody Warns You About

• Model Serving Architectures: From Prototype to Production

• LLM Safety and Guardrails in Production

• Hands-on: A/B Test Prompts in Production · Agent Guardrails and Safety · From Prototype to Product

Test you can run: in staging, turn off model access and use the feature. Whatever the user sees is your real failure behaviour. If it's a spinner, you have the same problem Penhale had.

Observe it once real users depend on it

Shipping isn't finished at launch. Model providers update models, prompts get edited, retrieval indexes go stale and user behaviour shifts. Each of these can quietly lower quality without raising an error. Production AI needs continuous evaluation on sampled live traffic, drift monitoring and observability that records why an agent did something, not just that it did it.

The worst failures are the silent ones, such as an agent reporting success on a task that actually failed. Those only show up if your monitoring checks results instead of trusting what the agent reports.

• Silent Hallucinations: When Your Agent Lies About Failures

• AI Agents Fail in Production in the Same Five Ways

• Hands-on: Continuous Evaluation in Production · Monitor for Prompt Drift · Production Observability for Agents · Cost-Monitor an LLM App

Test you can run: pick ten "successful" agent runs from last week and check the actual outcome in the downstream system. Your trusted success rate is the share that really succeeded.

Respond when the model does something unexpected

At some point it will. An AI incident is different from an outage: the system is up, but it's doing the wrong thing confidently. You need a way to contain it (a kill switch, falling back to the non-AI path, or tightening the confidence threshold), a way to find out how many users were affected, and a way to turn the failing case into a permanent eval case so it can't come back without being caught.

• AI Incident Response: When Your Model Does Something Unexpected

• AI Governance and Responsible AI. The governance pillar, for regulated and high-risk systems.

Test you can run: ask your team how long it would take to switch off the AI path for one feature without a deploy. If the answer involves a pull request, build the switch first.

Frequently asked questions

What does "shipping AI" mean? Shipping AI means getting an AI or LLM feature from a working prototype into production, where real users depend on it, and keeping it trustworthy once it's there. In practice that covers defining done (including failure behaviour), evaluating output, gating releases, staged rollout, monitoring for drift and responding to incidents.

Why do most AI projects fail to reach production? Usually not because of the model. The common causes are organisational and come from discovery: no owner for quality, no defined behaviour when the model fails, no eval set, and success criteria that only describe the demo. This article explains the mechanism.

How do you test an AI feature before release? Use layers. Deterministic checks and contract tests handle what can be checked exactly. A fixed eval set, scored by rules or an LLM-as-judge, handles output quality. Human review covers the highest-risk cases. Wire the evals into CI as a quality gate so a regression fails the build. Start with the test strategy.

What should an AI production readiness checklist include? At minimum: an owned and versioned eval set, an enforced threshold in CI, defined behaviour when the model is unavailable, a rollback that doesn't need a deploy, cost and latency budgets, live-traffic monitoring and an incident runbook. Here's the full checklist.

How do you monitor an LLM in production? Sample live traffic and score it continuously against the same rubric as your offline evals. Track drift in inputs and outputs, record agent decisions in enough detail to explain them, and alert on quality and cost as well as errors. Continuous evaluation in production walks through the setup.

Where to start

If you have an AI pilot that's "nearly ready", don't start with the model. Start with the readiness checklist and mark every item you can't show evidence for. That list is your real path to production. Most teams find the gaps aren't technical: they're the definitions, owners and failure behaviours that discovery skipped.

For the building side (prompting, RAG, agents and cost), see the companion guide, Building LLM Applications. For the release mechanics shared with all software, see Automation and Delivery.