Chain Prompts to Break Down Hard Tasks — Practical Prompt Engineering, Part 5

Shawn West
Jul 16
9 min read

Updated: Aug 17

Practical Prompt Engineering · Part 5

By now the support-ticket summarizer does one thing well. Real systems want more: read the ticket, decide its category, judge urgency, draft a reply, and flag anything that needs a human. The tempting move is to bolt all of that onto one giant prompt — "summarize, classify, draft a response, and tell me if it needs escalation." It'll appear to work in a demo. Then you'll notice the category is sloppy when the reply is good, or the escalation flag flips depending on how long the draft ran. One prompt juggling four objectives optimizes none of them, and worse, you can't tell which objective it dropped on any given call.

Chaining fixes that by turning one overloaded prompt into a short pipeline of focused steps, each doing a single job you can inspect. This tutorial builds that pipeline on top of the workspace from Part 1 — and, just as important, tells you when not to chain, because every hop you add costs latency, money, and reliability.

Before you start

Have the Part 1 workspace importable: run(system, user) returning (text, usage), the MODEL constant, log_run, and the cost() helper. Step 1 imports the first three; troubleshooting a cost jump needs the fourth.
Finish Part 4 first, or at least adopt its lesson. Chained steps pass JSON, not prose — run_json is the only new primitive in this tutorial, and it exists because stage two shouldn't be parsing stage one's paragraph.
Python and a working API key, with budget for three to four calls per ticket. Every hop is a full API call, and the validation stage in Step 6 adds a fourth.
A shell with grep. Step 7 debugs the chain by grepping prompt_log.jsonl on the step's system prompt; without the log there's nothing to grep and you're back to guessing which hop broke.
Two real tickets that aren't happy paths: one that should escalate (a legal threat, a security report, or an explicit request for a manager) and one thank-you note with no actual problem in it. The gates never prove themselves on a well-behaved ticket.
Block about an hour and a half — the labeled steps total 95 minutes.

What you'll build

A pipeline for support tickets with four stages: classify → (route) → summarize → draft reply → validate. You'll see how to pass structured output from one step into the next, how to branch so expensive steps only run when they're needed, and how to log every hop so a failure tells you exactly which step broke.

The reliability math nobody mentions (read this first)

Here is the argument for keeping chains short, in one number. Suppose each step in your chain is 95% reliable — it does its job correctly 19 times out of 20. That's a good step. Chain four of them and the end-to-end success rate is 0.95⁴ ≈ 81%. Chain six and you're at 0.95⁶ ≈ 74%. The failures compound because the whole chain is only as correct as the product of its parts, and one wrong hop poisons everything downstream. (This is the same trap behind the industry pattern where a 95%-reliable model still ships a ~60%-reliable system — chaining is where that decay happens.)

So the POV that runs through this whole tutorial: chain because the task genuinely has separable steps or needs a decision gate — never because chaining feels sophisticated. Every step you add has to earn its place against that 0.95 tax.

Step 1: Recap the primitives (5 min)

You already have what you need from Part 1: a run(system, user) that calls the model and returns text, a MODEL constant, and a JSONL log. Add one helper — a JSON-returning call, since chained steps pass structured data, not prose (that's the Part 4 lesson applied here):

import json
from workspace import run, MODEL, log_run   # from Part 1

def run_json(system: str, user: str) -> dict:
    """A step that returns structured data for the next step to consume."""
    text, usage = run(system + "\nRespond with JSON only.", user)
    log_run(system, user, text, usage)
    return json.loads(text)

Every stage below is just a function that takes the previous stage's output and returns the next.

Step 2: Stage one — a cheap classifier that routes (20 min)

The first stage is the most valuable, because it lets you avoid the expensive stages. Classify the ticket, and the classification decides what runs next:

CLASSIFY = """You are a support triage classifier. Given a ticket, return:
{
  "category": "billing" | "technical" | "account" | "other",
  "urgency": "low" | "medium" | "high",
  "needs_human": true | false
}
Set needs_human=true only for legal threats, security reports, or explicit
requests for a manager."""

def classify(ticket: str) -> dict:
    return run_json(CLASSIFY, ticket)

Run it on a real ticket:

ticket = ("Customer: I was double-charged for order #12345 — two "
          "$79.00 charges on the same card. I want this fixed today "
          "or I'm disputing it with my bank.")
print(classify(ticket))

{'category': 'billing', 'urgency': 'high', 'needs_human': False}

That one call is cheap — a few hundred tokens — and it's now the switch for the rest of the pipeline. A "dispute it with my bank" billing ticket routes differently from a "how do I change my avatar" ticket, and you spend the expensive model time only where it's warranted. This is the single highest-leverage reason to chain: not to do more work, but to skip work you don't need.

Step 3: Stage two — summarize, reusing what you built (10 min)

The summarizer is the sharpened prompt from Parts 2–3. In the chain it's just the second function, and it only receives what stage one already validated:

SUMMARIZE = """You are a support analyst. In one sentence, state the
customer's core issue and any dollar amounts or order numbers. If the
ticket contains no actual problem, respond exactly: "No issue to summarize."""

def summarize(ticket: str) -> str:
    text, usage = run(SUMMARIZE, ticket)
    log_run(SUMMARIZE, ticket, text, usage)
    return text.strip()

Step 4: Stage three — draft a reply from structured context (15 min)

Now the payoff of passing structured data down the chain: the reply-drafting step gets the category and urgency as facts, not something it has to re-derive from the raw ticket. It does one job — write the reply — and does it with the earlier decisions handed to it:

DRAFT = """You are a support agent. Write a 3-4 sentence reply.
You are given the ticket, its category, and its urgency. Acknowledge the
specific issue, state the next step, and match the urgency in tone (a
high-urgency billing dispute gets a faster, more concrete commitment).
Never promise a refund amount you weren't told to offer."""

def draft_reply(ticket: str, meta: dict, summary: str) -> str:
    context = (f"Ticket: {ticket}\nCategory: {meta['category']}\n"
               f"Urgency: {meta['urgency']}\nSummary: {summary}")
    text, usage = run(DRAFT, context)
    log_run(DRAFT, context, text, usage)
    return text.strip()

Notice what each step doesn't do. The classifier doesn't write prose. The drafter doesn't re-classify. Because each prompt has one objective, you can read its output and know immediately whether that step is the problem — which is exactly what the one-giant-prompt version made impossible.

Step 5: Wire the pipeline with a routing gate (20 min)

Assemble the stages, and put the routing decision from stage one to work. Tickets flagged needs_human skip the auto-reply entirely — you don't want a model drafting a breezy response to a legal threat:

def pipeline(ticket: str) -> dict:
    meta = classify(ticket)                       # stage 1: cheap, always runs

    if meta["needs_human"]:                        # gate: stop the chain early
        return {"route": "human", "meta": meta,
                "reason": "flagged for human review"}

    summary = summarize(ticket)                     # stage 2
    if summary == "No issue to summarize.":         # gate: nothing to reply to
        return {"route": "auto-close", "meta": meta, "summary": summary}

    reply = draft_reply(ticket, meta, summary)      # stage 3
    return {"route": "auto-reply", "meta": meta,
            "summary": summary, "reply": reply}

result = pipeline(ticket)
print(result["route"], "|", result["meta"]["urgency"])
print(result["reply"])

auto-reply | high
I'm sorry for the double charge on order #12345 — I can see two $79.00
charges. I've escalated this for a same-day reversal of the duplicate and
you'll get an email confirmation once it's processed. If your bank shows
the second charge tomorrow, reply here and I'll follow up directly.

Two gates, three model calls, and every branch does the least work it can. The needs_human ticket costs you one cheap classifier call instead of three; the thank-you note auto-closes after two. That's the pipeline paying for its own complexity.

Step 6: Validate the last step before you trust it (15 min)

A chain's output is only as good as its final hop, and the drafter is the step most likely to overpromise. Add a cheap validation stage — it's the reliability insurance for everything upstream:

VALIDATE = """You are a compliance checker. Given a drafted support reply,
return {"ok": true|false, "reason": "..."}. Set ok=false if the reply
promises a specific refund amount, a guaranteed date, or admits company
fault in a way that creates liability."""

def validate_reply(reply: str) -> dict:
    return run_json(VALIDATE, reply)

Fold it into the pipeline after draft_reply, and route any ok: false to a human instead of sending it. Yes, that's a fourth call and another 0.95 on the reliability product — but a validation step that only ever blocks is the cheapest possible insurance against the one draft in fifty that promises a refund you never authorized.

Step 7: Log each hop so failures are debuggable (10 min)

Because every stage already calls log_run, your JSONL log now contains a trace of the whole chain. When a result looks wrong three weeks from now, you don't guess — you read the log and see which step produced the bad output:

grep '"system":"You are a support triage' prompt_log.jsonl | tail -1

That single grep tells you whether the classifier mislabeled the ticket or the drafter went off-script. In the one-giant-prompt version, there was nothing to grep — the failure was smeared across a single opaque call.

What you just did

You turned one overloaded prompt into a four-stage pipeline where each step has a single job, structured data flows between steps, a cheap classifier routes work away from the expensive stages, and a validator guards the output. You also learned the tax: four steps at 95% each is an 81% system, so you keep chains as short as the task allows and gate aggressively so most tickets never touch every stage.

The pipeline, stage by stage

Stage	Its one job	Gate that can skip it
classify	Return category, urgency and needs_human	None — it's cheap, always runs, and it's the switch for everything after
summarize	One sentence naming the core issue plus any dollar amounts or order numbers	needs_human: true routes to a human after a single classifier call
draft_reply	A 3–4 sentence reply, handed category and urgency as facts it doesn't re-derive	"No issue to summarize." auto-closes the ticket after two calls
validate_reply	Return ok plus a reason; block refund amounts, guaranteed dates, and admissions of fault	Only runs on a draft that exists; ok: false routes to a human instead of sending

You're done when

pipeline(ticket) on the double-charge ticket returns route auto-reply with meta["urgency"] == "high", and the drafted reply names order #12345 and the two $79.00 charges — the classifier's decision visibly reached the drafter as a fact, not a re-derivation.
Your escalation ticket returns {"route": "human", ...} and the log shows exactly one model call for it, while the thank-you note auto-closes after two. If every ticket costs three calls, the gates are miswired.
grep '"system":"You are a support triage' prompt_log.jsonl | tail -1 returns the classifier's own entry from your last run, and swapping the system prompt in that grep pulls up each of the other stages separately. Every hop is findable on its own.

Troubleshooting

A wrong classification poisons everything downstream. The chain trusted stage one and it was wrong. Fix stage one first (better prompt, few-shot from Part 3), and add the needs_human gate as a safety net rather than relying on any single label.

End-to-end results feel unreliable even though each step looks fine. That's the 0.95ⁿ math, not a bug. Count your steps. If you have six, ask which two you can merge or drop.

Cost jumped when you added a stage. Every hop is a full API call. Profile with the Part 1 cost() helper and confirm the routing gates are actually short-circuiting — if needs_human never fires, the gate is miswired.

You can't tell which step failed. You're not logging per hop. Make sure every stage calls log_run, then grep the log by the step's system prompt.

Latency is too high. Chained calls are sequential — three hops is three round-trips, often 3–6 seconds. Run independent stages concurrently, or collapse two adjacent steps back into one if they don't actually need separating.

Common mistakes

Chaining for its own sake. If one prompt does the job at acceptable quality, one prompt is the right answer. The pipeline is for tasks that genuinely branch or need a gate.
Passing prose between steps. Stage two shouldn't parse stage one's paragraph. Pass JSON; that's what Part 4 was for.
No gates. A chain with no early exits runs every expensive step on every ticket, including the ones that needed a human on line one.
Trusting the last hop. The drafter is where overpromising happens; a validation step that only blocks is cheap insurance.
Ignoring the reliability product. Six 95% steps is a 74% system. Every added step is a real cost, not a free feature.

Continue the Practical Prompt Engineering path

Previous — Part 4: Structured Outputs (JSON and Beyond)
Next — Part 6: Prompting for Code Generation — put a chain to work generating and verifying code.