Thinking Tokens

Every article in this phase — Chain-of-Thought, Reasoning, Test-Time Compute, Inference-Time Scaling — has described a model working through intermediate steps, without naming the literal, concrete thing those steps actually consist of. That thing is thinking tokens.

The simple definition

Thinking tokens are the actual tokens a model generates as part of its internal, step-by-step reasoning process, before it produces its final, user-facing answer. Recall from the Token article that a model processes and generates text in these small units.

Thinking tokens are simply the tokens spent on the reasoning process itself — the “First, I’ll calculate X, then Y” content described conceptually in the Chain-of-Thought article, given a concrete, countable, billable form.

Why this needs to be a separate, named category of tokens at all

Recall from the Token article that API pricing and context length are measured in tokens — every single token counts, whether it’s part of the final answer or not.

Once reasoning models began generating substantial amounts of intermediate reasoning, as covered in the Test-Time Compute article, it became necessary to distinguish these reasoning tokens from ordinary output tokens — they consume real compute, real cost, and real context window space, exactly as covered in the Context Length article, but they aren’t necessarily meant to be the final, polished thing a user reads.

flowchart LR
    A[Prompt] --> B["Thinking tokens: 'First, let's consider... then...'"]
    B --> C["Final answer tokens: the polished response"]
    C --> D[What the user actually sees, by default]

ANALOGY vs. TECHNICAL REALITY

Analogy: Think of a mathematician’s scratch paper, covered in exploratory calculations, false starts, and intermediate work, sitting right next to the clean, final proof they eventually write up and present.

The scratch paper isn’t wasted effort — it’s genuinely where the real problem-solving happened — but what gets shown to a colleague reading the final result is usually just the clean, polished version.

Where this breaks down: A mathematician’s scratch paper is a physical object they can freely glance at or discard.

Thinking tokens are still real, generated content, produced through the exact same autoregressive process covered in the Autoregressive Generation article — and, critically, whether a user actually gets to see that “scratch paper” at all is a deliberate design decision made by the company building the model, not something automatically visible the way physical scratch paper would be.

Real, current, documented approaches to handling thinking tokens

This is worth grounding precisely, since different companies have made genuinely different choices here, and it’s useful to know both, concretely.

OpenAI’s o1 model, according to its own documentation, hides its full reasoning tokens from users by default, providing only a summarized version of the thinking process — a deliberate choice to keep responses cleaner while still giving some visibility into how the model approached a problem.

Anthropic’s Claude models, by contrast, expose an explicit thinking parameter with a configurable budget_tokens setting through their API, letting a developer set a specific token budget for Claude’s internal reasoning — and depending on the model version, either the full thinking content or a summarized version gets returned alongside the final answer, giving developers real, granular control over both the reasoning budget and its visibility.

A concrete example, layered

For a simple beginner example: asked “What is 17 times 24?”, a reasoning model might generate thinking tokens like “17 × 24 = 17 × 20 + 17 × 4 = 340 + 68 = 408” before outputting the final answer “408” — the thinking tokens contain the actual working-out, while the final answer tokens contain just the clean result.

For a production example: Anthropic’s documented Claude API lets a developer set a budget_tokens value — say, 12,000 — specifically allocating that many tokens for Claude’s internal reasoning process before it commits to a final answer, with the API’s own documentation noting that response times increase accordingly, and that streaming becomes required once total token limits exceed a certain threshold, exactly the kind of concrete, practical engineering trade-off covered throughout the Test-Time Compute article.

Why thinking tokens cost real money, even when hidden

It’s worth being direct about this, since it’s a genuinely important, sometimes surprising practical detail. Recall from the Token article that API pricing is charged per token processed.

Thinking tokens are real, generated tokens — they cost money and consume time, exactly like any other generated token, even when a company chooses not to display them to the end user by default.

This is precisely why reasoning models tend to be noticeably more expensive per response than standard models, as covered in the Test-Time Compute article’s cost discussion — a user might only see a short final answer, but the actual computation, and the actual bill, reflects the full reasoning process that produced it.

Count input, thinking, and visible output separately

Imagine a difficult request uses:

Input tokens supplied by the application       1,200
Internal reasoning or thinking tokens          2,000
Visible final-answer tokens                      300

The user may see only the 300-token answer, while usage, latency, context accounting, and billing can also involve the hidden reasoning work according to the provider’s rules. Thinking tokens are generated during inference; they are not model parameters and they do not become permanent learning after the request.

Current GPT and Gemini controls

OpenAI’s GPT-5.6 model page lists reasoning-token support and six reasoning-effort settings from none through max. Its current guidance says pro mode aggregates the tokens used by additional model work into reported usage and bills them at the selected model’s token rates.

Google’s current Gemini thinking documentation reports thinking usage separately from visible output usage. Gemini 3 models use thinking levels, while Gemini 2.5 models use a numeric thinkingBudget: the documented Gemini 2.5 Pro range is 128–32,768 thinking tokens, and Gemini 2.5 Flash supports 0–24,576. Availability, defaults, and whether thinking can be disabled differ by model.

flowchart LR
    I[Input tokens] --> M[Thinking/reasoning work]
    M --> O[Visible final answer]
    M -. provider may return a summary or signature .-> S[Reasoning metadata]

A reasoning summary is not necessarily the complete internal reasoning trace. Applications should present concise evidence, calculations, citations, or verification that helps users judge the result rather than treating hidden-token count as proof of correctness.

Real-world example: easy and hard questions use different effort

Easy: “Which planet is known as the Red Planet?”
Likely need: little thinking; answer “Mars.”
Hard: “Create a conflict-free timetable for 40 classes and explain any compromise.”
Likely need: planning, checking constraints, and possibly tools.

For supported GPT models, an application can select a reasoning-effort setting. For supported Gemini models, it can select a thinking level or model-dependent thinking budget. Those settings travel in the API configuration beside the prompt; they are not ordinary words inserted into the user’s message.

flowchart LR
    A[Prompt content] --> R[API request]
    B[Reasoning or thinking setting] --> R
    C[Sampling and output settings] --> R
    R --> M[Model performs internal work]
    M --> U[Usage counts thinking/reasoning tokens]
    M --> F[User receives final answer or summary]

As of August 2026, GPT-5.6 documents reasoning effort from none to max. Gemini documentation distinguishes thinking-level controls for Gemini 3 models from numeric thinking budgets for Gemini 2.5 models. Applications should query or read the exact model documentation because supported settings and defaults change between versions.

Common misconception

A frequent beginner assumption: that if a model doesn’t show its reasoning, it didn’t actually reason through the problem at all — that hidden thinking tokens mean no real intermediate computation happened.

As OpenAI’s o1 documentation makes clear, this isn’t accurate — the reasoning tokens are genuinely generated and genuinely influence the final answer, exactly as covered in the Chain-of-Thought article’s explanation of why written-out steps do real computational work; they’re simply not displayed to the user by default, a visibility choice, not an indication that no reasoning occurred.

Closing out this phase

This article completes the Prompting & Reasoning phase, and it’s worth tracing the full arc it built: a Prompt starts every interaction, shaped by any standing System Prompt, with Prompt Engineering as the deliberate practice of crafting both well — through Zero-Shot instructions, Few-Shot examples, or Chain-of-Thought step-by-step elicitation. Temperature, Top-k, and Top-p control how each individual token actually gets chosen.

And Reasoning, powered by Test-Time Compute and explained by Inference-Time Scaling, all comes down to this article’s concrete mechanism — thinking tokens, the literal, billable, sometimes-hidden units of computation where a model’s actual problem-solving happens.

From here, the glossary is positioned to move into the specialized techniques — fine-tuning, RAG, and agentic systems — built on top of everything covered so far.

In one sentence

Thinking tokens are the literal, generated tokens a model spends on internal reasoning before producing its final answer — real, billable, and genuinely load-bearing to the computation, whether a specific product chooses to show them to you in full, summarize them, or hide them entirely.

Related Terms

• Chain-of-Thought
• Reasoning
• Test-Time Compute
• Token
• Context Length