When asking for intermediate steps helps, when it hurts, and how to keep the reasoning checkable.
Asking a model to work through a problem in stages often improves accuracy on tasks that need several dependent judgements: arithmetic with units, multi-condition rules, comparisons across documents. The same technique can also produce confident, fluent reasoning that is wrong at step two and never recovers. The difference is whether the steps are checkable.
Why steps help
Some tasks cannot be done in one leap. Deciding whether a refund is allowed needs the policy, the order date, the delivery date and the exception list, in that order. A single-shot answer compresses those judgements into one token prediction, and compression loses the constraint that failed. Forcing the intermediate results into the output keeps each judgement visible, and visible judgements can be checked one by one.
"Think step by step" is weak because it requests a style of prose. Ask for the specific intermediates instead: "list the policy clauses that apply, then the dates, then the decision, then the amount." That request produces a structure you can audit. It also bounds the reasoning: a model told to list clauses first is less likely to invent an exception later.
Example: "First list each condition from the policy and whether the order meets it. Then state the decision. Then state the refund amount. Keep each section to one line."
Separate reasoning from the answer
If the output feeds a system, ask for the final answer in a fixed field and the reasoning in another. The reasoning helps a human reviewer; the field helps the code. Mixing them means a parser has to guess where the answer starts, and it will guess wrong on the day a sentence begins with a number. Two fields also make it obvious when the reasoning and the answer disagree.
Verify the chain, not the confidence
A step-by-step answer reads as more trustworthy than a short one, which is a trap. Fluency is not evidence. Check each intermediate against its source: does the policy clause exist, is the date parsed correctly, does the arithmetic hold. Where a step cannot be verified, treat the conclusion as a draft. The technique raises the ceiling on accuracy; it does not remove the need for a check.
When not to ask for steps
For extraction, classification and rewriting, intermediate reasoning usually adds noise. The task is the output, and a preamble makes the result harder to parse. Use steps when the task has dependent judgements and a wrong middle step would be invisible in the final answer. Otherwise, ask for the result and move on.
Checklist
- Does the task have dependent judgements?
- Are the intermediates named, not just requested?
- Is the final answer in its own field?
- Can each step be checked against a source?