Glossary
What is test-time compute?
In short
Test-time compute is the extra calculation a model spends while it answers, for example on reasoning or on checking several options, to solve a task more accurately.
Language models · test-time compute
MMOOZZGG.com editorsChecked: 1 min read
For a child
In a test, you solve an easy example at once, but you sit longer over a hard problem: you try one way, then another, and check the answer. The more time you spend on a hard problem, the more often you get it right. New neural networks can also think longer about a hard question before they answer.
In plain words
Earlier, a model answered in one pass: however hard the task, it spent about the same effort on it. The new approach gives it time to think. The model can first write out the solution step by step, generate several answers and pick the most common one, or check a draft and fix it. All of this happens after training, right during your request. For example, on a hard task the model may write a long draft of reasoning but show you a short result.
For business
The approach changes the economics of AI. Before, you bought quality with model size; now you can also buy it with time and tokens for a specific request. The hidden drafts are paid for: at Anthropic, for example, thinking tokens count as billed output and toward the max_tokens limit. Two things follow. A complex contract review or calculation becomes more accurate, but more expensive and slower. Request sorting and other simple tasks gain nothing from long reasoning: you pay for extra work. A practical setup splits the flow: bulk requests go to a fast model, hard ones go to reasoning mode with a capped budget. Compare token prices across vendors in advance, before you choose the setup.
For a specialist
Scaling test-time compute, also called inference-time compute, raises quality through computation at answer time, not at training time. Snell et al. (2024) name two mechanisms: adaptively changing the answer distribution (step-by-step revision of a draft) and searching among candidates checked by a process reward model as verifier. They showed that spreading the budget by query difficulty gives more than a fourfold efficiency gain over best-of-N. With equal FLOPs, on tasks where a small model already sometimes succeeds, it can beat a model 14 times larger. The simplest variant, self-consistency (Wang et al., 2022), samples several reasoning chains and takes the most common answer, which gave +17.9% on GSM8K. In an API this is the thinking budget: at Anthropic the budget_tokens parameter is at least 1024 tokens, and in newer models an adaptive mode with an effort level has replaced it.
Example
The finance team asks a model to check the tax calculation for a complex deal and turns on reasoning mode. The answer comes noticeably later and costs more, but the model finds the error in the rate by itself.
Related terms
Questions and answers
Should you always turn on the reasoning mode?
Sources
- Snell et al., Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters, 2024arxiv.org
- Wang et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models, 2022arxiv.org
- Anthropic, Extended thinking, Claude API documentation, checked 16 October 2026platform.claude.com
Not sure whether your company needs this?
AI solutions map