Reasoning models explained: what thinking out loud actually buys

What does a reasoning model actually do that a normal one doesn’t? It thinks before it answers, and you pay for that thinking whether you see it or not.

The mechanism is simpler than the marketing suggests, and understanding it tells you exactly which of your tasks will benefit.

The idea started as a prompting trick

Before it was an architecture, it was a way of asking. Google Research showed in Chain-of-Thought Prompting Elicits Reasoning in Large Language Models that “prompting a 540B-parameter language model with just eight chain of thought exemplars achieves state of the art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier.”

Read that comparison again, because it’s the whole origin story. Eight examples in a prompt beat a model that had been fine-tuned for the task.

The intermediate steps weren’t decoration. They were where the computation happened, and writing them out gave the model somewhere to do the work.

That finding sat on top of an earlier one. The GPT-3 paper, Language Models are Few-Shot Learners, had already shown models picking up tasks from prompt examples “without any gradient updates or fine-tuning”, so the surprise wasn’t that examples worked. It was that showing the working mattered more than showing the answer.

Why writing steps helps at all

A transformer does a fixed amount of computation per token generated. That’s a hard constraint set by the architecture, described in Attention Is All You Need.

So a problem needing more computation than one token’s worth has only one place to put it: more tokens. Generating intermediate steps is literally buying more compute for the same question.

Thinking out loud isn’t a stylistic choice. It’s the only way a transformer can spend more effort on a harder problem.

Then it moved into training

If step-by-step working helps, the obvious next move is to train models to do it by default rather than asking each time. That’s what a reasoning model is.

DeepSeek’s R1 paper reports the striking version of this: reasoning behaviour emerging from reinforcement learning alone, “obviating the need for human-labeled reasoning trajectories”, with “self-reflection, verification, and dynamic strategy adaptation” appearing without being demonstrated.

Nobody showed the model how to check its own work. It found that behaviour because checking improved the reward.

The word that decides everything is verifiable

The R1 results come from mathematics, coding competitions and STEM problems, and those share one property. You can check the answer mechanically.

That’s what makes the training work. The reward can come from the world rather than from a human rater, which is a far sturdier signal than the preference data used in ordinary preference tuning.

It also explains the shape of the gains. Reasoning models improve most where correctness is checkable and least where judgement is the product.

Which is why the improvement feels uneven in practice. A model that solves competition maths reliably can still write a mediocre summary, because nothing in its reasoning training rewarded good summarising.

The R1 work was later published in Nature, volume 645, which is unusual for this field and a reasonable signal that the result survived scrutiny.

What you actually pay for

	Standard model	Reasoning model
Tokens billed	Answer only	Thinking plus answer
Latency	Immediate	Seconds to tens of seconds
Best at	Transformation, extraction, drafting	Multi-step problems with checkable answers
Worst at	Long chains of dependent steps	Anything needing a fast reply

Reasoning tokens are billed as output, and output is the expensive half. On the current spread, output runs $6 to $30 per million.

A model that thinks for two thousand tokens before writing two hundred has charged you for eleven times the visible result. Worth it on a hard problem, absurd on an easy one.

Latency is the other cost, and it’s the one users feel. Several seconds of silence before a reply is fine for a research task and unacceptable in a chat interface, so the model choice is partly a product decision rather than a purely technical one.

Where the industry shift comes from

This didn’t arrive because researchers found it elegant. It arrived because the other lever got expensive.

Epoch AI reports frontier training compute growing 5x per year since 2020. When each generation of pre-training costs an order of magnitude more, spending compute at request time instead becomes an attractive second axis.

DeepMind’s compute-optimal work had already told labs how to divide a training budget. Inference-time reasoning added somewhere else to spend entirely, which is part of the argument about whether scaling stalled.

It also moves the cost from the lab to you. Training is their bill, inference is yours, on every request, forever.

The sceptical reading

Not everyone accepts that reasoning models reason, and the objection is worth taking seriously.

On this view the model is doing the same next-token prediction as before, just over a longer sequence that happens to include useful intermediate states. Nothing qualitatively new is happening, and calling it reasoning imports a claim the evidence doesn’t support.

There’s also a measurement worry. Reasoning models are evaluated on precisely the verifiable tasks they’re tuned for, so published gains overstate what a mixed workload sees, which is a specific case of why benchmark scores mislead.

The counter is that the mechanism doesn’t much matter if the answers are better. Whether extra tokens constitute thought is a philosophical question. Whether they raise accuracy on hard problems is an empirical one, and the answer there is yes.

What would change the picture

Evidence of large gains on unverifiable tasks would be the significant development, because it would mean the technique generalises beyond the domains where a checker exists.

Watch also whether thinking budgets become controllable per request. Being able to say how hard to think would turn a blunt model choice into a dial, which is what most production systems actually want.

Until then the practical rule holds. Route verifiable multi-step work to a reasoning model, send everything else to a fast one, and measure the split rather than guessing at it.