Why the Smartest AI Models Now “Think” Before They Answer (And When It’s a Waste)

If you have used a recent AI model and seen a little “Thinking…” indicator, sometimes for many seconds, you have met the biggest shift in how language models work since ChatGPT launched. These are reasoning models, and they changed the AI race by changing what “smarter” means.

The old way: answer immediately

Classic chatbots generate a reply one token at a time, left to right, starting right away. That works well for fluent writing and recall, but it is a bad fit for problems that require working through steps, like a tricky math problem or a bug that hides three functions deep. The model has to get the first words right without having worked anything out.

Researchers noticed early that you could help by asking for the work. A 2022 Google paper, “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models”, showed that prompting models to write out intermediate steps significantly improved results on arithmetic, commonsense and symbolic reasoning tasks. “Let’s think step by step” became one of the most famous phrases in AI.

The new way: train the model to think

Reasoning models take that idea and build it in. Instead of relying on a prompt trick, the model is trained, largely with reinforcement learning, to produce a long internal chain of reasoning before giving a final answer, and it is rewarded when that reasoning leads to correct results.

OpenAI brought this into the mainstream with o1 in September 2024. In its technical post, “Learning to reason with LLMs”, OpenAI reported a striking result on the AIME 2024, a hard high school math competition: GPT-4o solved about 12% of problems on average, while o1 averaged 74% with a single attempt and 83% when using consensus across 64 samples.

OpenAI also described two scaling trends: performance improved with more reinforcement learning during training, and with more time spent thinking at answer time. That second idea, often called test-time compute, was new. It meant you could get a smarter answer not only by building a bigger model, but by letting the same model think longer.

The open-source moment

In January 2025, DeepSeek released R1, an open-weight reasoning model with performance comparable to o1 on many benchmarks, and published a detailed paper on how it was trained. One of its most discussed findings came from a variant called R1-Zero, trained with reinforcement learning without supervised fine-tuning first. The researchers reported that the model learned on its own to spend more time on harder problems and to revisit and correct its earlier steps, including a moment they described as an “aha moment.”

That openness mattered. Within months, reasoning modes became standard across the industry, in closed models from the major labs and in open models anyone could run.

What reasoning is good for

Thinking before answering pays off most on problems with a clear right answer and many steps:

• Math and science problems with multi-step derivations
• Coding, especially debugging and algorithm design
• Planning, like breaking a large task into ordered steps
• Analysis of long documents where details must be cross-checked
• Agents, which need to decide what to do next over many steps

For quick factual questions, casual writing or brainstorming, extra thinking often adds delay without adding much quality.

The costs and caveats

It is slower and pricier. Reasoning tokens are generated and, in most APIs, billed even if you never see them. A hard question can consume many times the tokens of a normal reply. Many tools let you choose a reasoning effort level for this reason.

Visible reasoning is not a window into the mind. The summaries some apps show are not guaranteed to reflect exactly how the model reached its answer. Researchers are actively studying how faithful these chains of thought are.

Thinking has limits. A June 2025 paper from Apple researchers, “The Illusion of Thinking,” tested reasoning models on puzzles of controlled difficulty and reported that accuracy collapsed beyond certain complexity levels. Other researchers disputed parts of the setup, and the debate itself is a healthy reminder: reasoning models are much better at structured problems, not infallible.

They can still be confidently wrong. A long, articulate chain of reasoning can make a wrong answer more persuasive. Check important results.

How to get the most out of a reasoning model

1. Use it for hard problems, and a fast model for easy ones.
2. Give the full problem up front: constraints, examples, what a good answer looks like.
3. Ask for verification, like tests for code or a check of the final number.
4. Do not over-prompt the steps. These models plan on their own; clear goals beat micromanaged instructions.

The bottom line

Reasoning models taught AI to slow down, and that made it dramatically better at the kind of work that used to expose its weaknesses. They also introduced a new lever: spend more compute at answer time, get a better answer. Understanding when that trade is worth it, and when a quick answer is fine, is now one of the most practical AI skills you can have.

Sources

• Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” (arXiv:2201.11903), 2022
• OpenAI, “Learning to reason with LLMs”, Sept. 12, 2024
• DeepSeek-AI, “DeepSeek-R1” (arXiv:2501.12948), Jan. 2025
• Apple Machine Learning Research, “The Illusion of Thinking”, June 2025