The Scalability article closed on a promise: the precise economic unit growth and infrastructure costs ultimately get measured in. This article covers exactly that: cost per token.
The simple definition
Cost per token is the price a provider charges for each individual token processed by a model, typically quoted separately for input tokens (what the user sends) and output tokens (what the model generates), usually priced per million tokens. Recall from the Token article, back in the Data Representation phase, that a token is the basic, discrete unit a language model actually processes. Cost per token is simply the real, dollar price attached to that exact unit — the specific number that determines what running any AI application actually costs in practice.
Why this specific, tiny unit became the field’s standard pricing currency
Recall from the Batch Inference and Quantization articles, back in the Infrastructure & Serving phase, that a model’s real, underlying cost comes from the actual GPU computation each request requires. Since that computation scales directly with how many tokens are processed, pricing per token became the natural, transparent way for providers to charge — a short question costs less than a long document, and generating a lengthy response costs more than a brief one, with the price scaling in direct, predictable proportion to the actual computational work performed.
flowchart LR
A[User sends a prompt: input tokens] --> B[Priced per million input tokens]
C[Model generates a response: output tokens] --> D[Priced per million output tokens, typically higher]
B --> E[Total cost = input cost + output cost]
D --> EThe real, dramatic price collapse this specific unit has undergone
This deserves to be told with real, precise numbers, because it’s one of the most striking cost trends in this entire glossary’s technology history. OpenAI’s original GPT-3, released via API in 2020, priced its largest, most capable model at roughly 20 per million tokens.
By July 2024, OpenAI’s GPT-4o mini launched at 0.60 per million output tokens — while scoring 82% on the MMLU benchmark, genuinely outperforming GPT-3.5 Turbo, the model it replaced. That’s roughly a 130-fold reduction in input token price over four years, for a model that was, by every published benchmark, considerably more capable than the one it was replacing.
OpenAI CEO Sam Altman described this exact trend directly, calling GPT-4o mini “a step towards intelligence too cheap to meter” — a genuinely striking phrase capturing just how fast this specific number has fallen.
ANALOGY vs. TECHNICAL REALITY
Analogy: Think of the historical price of computer memory or long-distance phone calls — both once counted among a household’s most carefully rationed expenses, now so inexpensive per unit that most people never think about them at all. Cost per token, this article argues, is following a genuinely similar trajectory, from a real, carefully-budgeted expense to something increasingly taken for granted.
Where this breaks down: Memory and phone calls fell in price primarily through decades of steady, gradual hardware improvement. Token pricing’s collapse, as covered throughout this phase, comes from several genuinely distinct forces converging at once — the quantization techniques covered throughout the Infrastructure & Serving phase, architectural efficiency gains like the Mixture of Experts approach covered in the Advanced Architectures phase, and, as covered in the MoE article’s own DeepSeek story, real, direct competitive pressure — a faster, more compressed timeline than most historical hardware cost curves.
A concrete example, layered
For a simple beginner example: a customer-support application processing 10,000 short conversations a day, at roughly 500 tokens each, would have cost real, meaningful money to run on GPT-3’s original 2020 pricing, but costs a small fraction of that on today’s cheaper, more capable small models — a genuine, direct illustration of why applications that were financially impractical in 2021 are routinely built today.
For a production example: real, published pricing pages from OpenAI, Anthropic, and Google all list separate, transparent per-million-token rates for each of their models, and developers building real production applications, echoing the Batch Inference article’s own real 50% discount structure, actively choose between models partly based on this exact, precise cost-per-token trade-off against capability.
Why lower cost per token doesn’t automatically mean lower total cost
It’s worth being honest about a real, important nuance here, connecting directly to the final article of this entire glossary. A dramatically cheaper per-token price can still produce a larger total bill if an application’s actual token usage — covered fully in the next article — grows even faster than the price falls, which is precisely why cost per token is only half of the real, complete economic picture a team actually needs to track.
The real, 2025 price war that made this number a matter of national headlines
This deserves telling with its own precise, current numbers, because it’s the single most dramatic recent chapter in this article’s larger price-collapse story.
When Chinese startup DeepSeek released its R1 model in January 2025, its published API pricing was startling by comparison to the rest of the market: roughly 0.55 per million input tokens and 3 to $15 per million tokens for comparable capability at the time — a real, documented gap of 10 to 30 times.
DeepSeek then went further, introducing off-peak pricing in February 2025 that cut costs by up to 75% during specific hours, a genuine, direct pressure campaign on rivals’ pricing worldwide.
Tim Sanders, VP of Research Insights at G2, framed the moment using Clayton Christensen’s “Innovator’s Dilemma”: established leaders often build solutions “almost too sophisticated and expensive,” creating real vulnerability to disruption from below — comparing DeepSeek’s arrival to how YouTube’s lower-quality but far more accessible video once upended traditional television.
A complete application bill
The model bill is only one part of an AI application’s cost. Imagine that 100 support tasks produce the following monthly totals using the GPT-5.6 Terra short-context standard rates stated above.
| Cost item | Example calculation | Cost |
|---|---|---|
| Uncached input | 500,000 × $2 / 1,000,000 | $1.00 |
| Cached input | 1,000,000 × $0.20 / 1,000,000 | $0.20 |
| Output | 200,000 × $12 / 1,000,000 | $2.40 |
| Search or other paid tools | Illustrative total | $0.80 |
| Human review | 25 reviews × $0.20 | $5.00 |
| Total | $9.40 |
If only 90 tasks finish successfully, the useful cost is $9.40 ÷ 90 ≈ $0.104 per successful task, not $9.40 ÷ 100. Retries, failed agent loops, safety checks, storage, GPUs, networking, and engineering may add more.
The tool and human-review prices in this example are illustrative; actual contracts differ. Cached input is cheaper in this dated model example, but it is not free, and a provider’s cache eligibility rules decide which repeated prefixes receive that rate.
Common misconception
Calculate one request
Use an illustrative model priced at 12 per million output tokens. A request uses 8,000 input tokens and 2,000 output tokens:
input cost = 8,000 / 1,000,000 × $2 = $0.016
output cost = 2,000 / 1,000,000 × $12 = $0.024
total model-token cost = $0.040The output contains fewer tokens but costs more because its rate is higher.
A dated current pricing example
On OpenAI’s pricing page accessed in August 2026, GPT-5.6 Terra standard short-context pricing is listed at 0.20 per million cached-input tokens, and $12 per million output tokens. Long-context and faster service tiers use different rates.
Pricing changes. Keep the formula in application code, but load or configure current rates from the provider rather than copying this dated example permanently.
The API bill includes more than ordinary text
Depending on the provider and model, costs may include cached input, cache writes, reasoning tokens, images, audio, search, code execution, storage, or other tools. Self-hosted open models replace token invoices with GPU rental, power, engineering, and operations costs.
Cost per successful task
A cheap model that succeeds only 70% of the time may require retries or human correction. Track:
cost per successful task
= total model + tool + infrastructure + review cost
divided by successful completed tasksVerified sources
A frequent beginner assumption: that a lower cost-per-token model is automatically the more cost-effective choice for any given application. As the GPT-4o mini example demonstrated, price and capability need to be weighed together — a slightly more expensive model that solves a task correctly on the first attempt can genuinely cost less overall than a cheaper model that requires several retries or produces lower-quality output requiring human correction afterward.
Where this fits in what comes next
You now understand the precise, per-token price everything in this glossary ultimately gets billed against. The final article in this entire glossary, Token Usage, covers the other, equally necessary half of the real cost equation — not the price per unit, but how many of those units an application actually consumes.
In one sentence
Cost per token is the price charged per unit of text a model processes, and the field’s real, roughly 130-fold price collapse — from GPT-3’s original 2020 pricing down to GPT-4o mini’s 2024 rates, for a genuinely more capable model — is precisely what Sam Altman meant by “intelligence too cheap to meter,” a real, ongoing trend reshaping which AI applications are financially viable to build at all.
Related Terms
- Author
- TechByteByByte Editorial Team
- Reviewed by
- TechByteByByte Admin
- Published
- Last reviewed