Previous lesson
What is frontier model serving cost
Continue the Model Costs track.
← Back to Compute College
Compute College
Why Reasoning Models Cost More to Serve
Reasoning models generate long chains of thought before answering, multiplying output tokens — and output tokens drive inference cost.
Compute & Pricing LessonsLearning path
One concept connected to AI compute market decisions.
5-8 minutesRead time
A practical introduction designed to be completed in one sitting.
Reasoning / Inference / TokensTags
Useful for founders, product managers, analysts, and investors.
Interactive calculator
Serving cost calculator
Total API calls served per month.
Average prompt size sent per request, in tokens.
Average response size generated per request, in tokens.
Price charged per million input tokens.
$
Price charged per million output tokens.
$
5,000,000 requests/month → estimated $42,000 in monthly serving cost.
Monthly input token cost$12,000
Monthly output token cost$30,000
Total monthly serving cost$42,000
Cost per 1,000 requests$8.40
Starting values are illustrative defaults you can edit — not live ComputeTape benchmark prices. Replace them with a real quote.
Plain-English definition
Plain-English definition
A reasoning model spends extra compute at the time you ask the question — generating a long internal chain of thought before it produces a final answer. That extra thinking is made of output tokens, and output tokens are the costly part of serving a model, so the same question answered by a reasoning model can consume several times the compute of a standard model. This use of extra inference-time compute to improve answers is often called test-time compute.
Why it matters
Why it matters
Serving cost scales with the tokens a model generates, not just the prompt you send in — and reasoning models generate far more of them. That shifts compute demand from one-time training toward recurring inference and changes how buyers plan capacity.
- Serving cost scales with generated tokens, not just the prompt, and reasoning models generate far more of them.
- It shifts demand from one-time training toward recurring inference, changing how buyers plan capacity.
- Longer outputs hold GPU memory longer per request, which lowers how many requests a cluster can run at once.
Simple example
Simple example
Suppose a standard answer is 500 output tokens, and the same question from a reasoning model produces 5,000 output tokens — 500 of answer plus 4,500 of "thinking." That is 5,000 ÷ 500 = 10× the output tokens for one answer, so if output tokens dominate the bill, that query can cost on the order of ~10× more to serve.
- Reasoning multiplies the expensive side of the bill: output tokens, not the prompt.
- A 10× jump in output tokens can mean roughly a 10× jump in serving cost for that query.
- Figures here are illustrative calculations, not quoted prices; real ratios vary by model, prompt, and how much reasoning a task triggers.
Example figures are illustrative calculations, not current quoted market prices.
Market signal
How to read the market signal
A jump in reasoning-model usage is an inference-demand signal, not a training-demand one. It pulls on serving capacity, high-bandwidth memory, and per-token economics rather than on one-off training clusters. Watch whether providers price reasoning or output tokens at a premium, and whether usage of long-output features is rising — both point to inference becoming the larger share of compute spend.
- Treat rising reasoning usage as recurring inference demand, not a one-time training build.
- Premium pricing on output or reasoning tokens is a direct signal of where serving cost concentrates.
- Growing long-output usage can tighten serving capacity and HBM even without more users.
Market read: reasoning usage turns better answers into recurring inference demand. Evidence discipline: token-cost ratios and "thinking length" are model- and task-specific, so record the model, the date, and the source for any cost-per-token figure, and keep illustrative ratios separate from observed prices.
Common mistake
Common mistake
Treating a reasoning model's cost like a standard model's because the prompt is the same size. The prompt (input) is processed in parallel and is comparatively cheap; the cost lives in the output, which is generated one token at a time and grows the memory (KV cache) held per request. A short question can still produce a long, expensive answer.
Practical takeaway
What you can do with this
Estimate reasoning-model cost from expected output length, not prompt size, and decide per task whether the extra reasoning is worth it.
- Buyers: budget on output tokens × output rate; ask providers for the output-token price specifically, and whether reasoning tokens are billed.
- Founders and analysts: separate "standard" from "reasoning" traffic in your unit economics, because blending them hides the real cost driver.
- For example, an illustrative reasoning query of 5,000 output tokens at an illustrative $15 per million output tokens is about $0.075 per query, versus ~$0.0075 for a 500-token standard answer — before input, overhead, or latency-driven capacity costs.
- Route easy tasks to a standard model and reserve reasoning for tasks that measurably benefit; the cheapest reasoning token is the one you did not need to generate.
- Keep provider observations separate from these calculated estimates: a modeled per-query cost guides decisions but is not an observed market price.
Decision check: more reasoning is worth paying for only when the better answer changes the outcome enough to justify the extra tokens.
Helpful memory trick
Helpful memory trick
Thinking out loud isn't free — every word of a model's reasoning is an output token, and output tokens are what you pay for.
Compute College
Follow the market after the calculation
Get the ComputeTape Morning Brief for daily AI compute pricing, power, capacity, and infrastructure signals.
Compute College track
Model Costs
Continue this Compute College lesson path
Next lesson
What is cost per million tokens
Continue the Model Costs track.