Better Prompting: Getting More from AI Tools

Everyone iterates on prompts. Almost nobody keeps anything from it — which is why the same team solves the same prompt four times and gets faster at none of them.

Penhale Mutual's claims team built a prompt for extracting the key facts from a loss adjuster's report. It took about nine attempts, and it ended up good.

Two months later a different handler needed the same thing for a slightly different report type and started over. Nine attempts again. So did the third. By the fourth, somebody thought to ask what had actually been learned, and the honest answer was: nothing transferable. Each round had converged, and each had thrown away the convergence.

The reason was visible in the chat logs. Every attempt rewrote most of the prompt. Attempt four changed the role, the format instruction and two constraints at once, and the output got better — so the next attempt kept all four changes and altered three more. Nobody could say which change had done the work, which meant nobody could carry it anywhere.

The unit is the loop, not the prompt

The prompt itself is a specification — a set of decisions you made instead of delegating. This post is about the process that gets you there, and it has its own failure modes that have nothing to do with wording.

The loop is four moves, and only the first is the one people do well:

Move	What it is	Done badly	Done well
Read	Diff the output against what you wanted	"It's not quite right"	Name the specific missing or unwanted thing
Attribute	Say which absent decision caused it	Blame the model	Point at the element that didn't cover it
Change	Alter the prompt	Rewrite half of it	Change one thing
Keep	Record what worked	Close the tab	Save the prompt and the reason

Move three is where the learning is created or destroyed, and move four is where it's banked or lost. Penhale was doing one and three, energetically, and neither two nor four.

The test you can run: open your last prompt-tuning session and count the changes between two consecutive attempts. If it's more than one, you learned that the pair was better, not which half.

Change one thing, or the improvement doesn't transfer

This is unglamorous and it's the whole discipline.

When you change four things and the output improves, you've found a bundle that works for this task. You have not found out why, which means you can't apply any part of it to the next task, can't explain it to a colleague, and can't tell later which parts have stopped earning their place. Prompts accumulate this way — every production prompt has a few lines nobody dares remove because nobody knows what they do.

One change per attempt is slower per attempt and much faster overall, because each attempt produces a fact. Penhale's fourth round did it deliberately and needed six attempts rather than nine, but the difference wasn't the count. It was that they came out knowing the role changed the tone and did nothing for accuracy, the worked example fixed the format entirely, and the two constraints they'd been carrying since attempt one had never done anything at all.

That last one is the recurring finding. Removing a line and seeing no change is a result, and it's the only way a prompt gets shorter.

The test: take a production prompt, delete one line you're unsure about, and run it on ten inputs. If nothing changes, you've found a line that was costing tokens and attention and buying nothing.

The output is evidence about your prompt, not about the model

The instinct on a bad output is to judge the model. The more useful reading is that the output is a fairly precise report on what your prompt actually said.

Too long means you didn't bound it. Hedged means you didn't say what to do when unsure. Wrong emphasis means you named an operation rather than a purpose. Restating your question back at you means the default genre of the material it learned from does that and you didn't say otherwise.

Read that way, the second move stops being guesswork. Rather than how do I make this better, the question is which decision, left open, produced exactly this. That question has an answer often enough to be worth asking first every time.

The test: for your last unsatisfying output, write the sentence you'd add to the prompt to prevent it. If you can't write one sentence, the problem isn't the prompt — it's that you don't yet know what you wanted.

The artifact is a template, not a good answer

Here's the part Penhale was missing entirely. The purpose of the loop, for anything you will do more than twice, is not the output. It's the prompt.

A team that finishes with a great extraction and an unsaved chat has done the work and kept none of it. A team that finishes with a saved prompt, a note on what each block is for, and two examples of inputs it handles badly has built a reusable asset — and the note about what it handles badly is the more valuable half, because it tells the next person where the edges are without them having to rediscover them.

This is also where prompting stops being a personal skill and becomes a team one. The saved prompt is the mechanism by which the fourth person doesn't spend nine attempts.

The test: count the prompts your team has written down somewhere shared, versus the number of recurring tasks it uses AI for. A large gap means everyone is re-deriving the same thing privately.

Knowing when to stop, and when to stop prompting altogether

The loop needs a stopping rule or it runs forever, and it needs an exit for the cases it can't solve:

• Stop when the next change is about taste. Once the remaining objections are preference rather than correctness, you're done — further iteration is just moving the output around.

• Stop when you can't tell if it improved. Past a point, single outputs stop distinguishing prompt versions, and the honest next step is measuring it on a set rather than reading one more result and forming an impression.

• Leave the loop when the information isn't there. No prompt reports a fact the model was never given. If the gap is a missing input, that's a scope and retrieval question, and continuing to iterate produces increasingly confident inventions.

• Leave when the variance is the problem. If the same prompt gives materially different answers, wording isn't the lever, and one run stopped being evidence a while ago.

• Don't loop on a one-off. For something you'll do once, a good-enough output in two attempts beats a perfect template you'll never open again. The loop is an investment; check there's a second occasion before making it.

The test: name what would have to be true for your current prompt version to be worse than the previous one. If you have no way to tell, you've been iterating on impressions for a while.

What to change this week

Don't tune a prompt. Instrument one round.

Next time you're refining a prompt, keep a three-column note: what you changed, what you expected, what actually happened. One change per row. It costs almost nothing and it converts an afternoon of fiddling into a short list of facts about your task — including, usually, one line that turns out to do nothing.

Then save the result somewhere the next person will find it, with two inputs it handles badly written underneath. That's the whole artifact.

Penhale keeps its extraction prompt in the repo beside the code that calls it, with a comment per block and four known-bad inputs in a file next to it. The fifth report type took two attempts. Not because the prompt was reusable — it wasn't, quite — but because the notes said which parts had ever mattered.