Logits

The Next-Token Prediction article described the model calculating “a probability for every possible next token.” This article covers exactly what happens one step before that — the raw numbers the model actually computes first: logits.

The simple definition

Logits are the raw, unnormalized numbers a model’s output layer produces for each possible token, before they get converted into actual probabilities. Recall from the Output Layer article that a multi-class classification output layer has one node per possible category, each producing its own raw value. For a language model, that means one node per possible token in the entire vocabulary — recall from the Token article that this vocabulary is roughly 100,000 to 200,000 tokens for OpenAI’s models. Logits are the raw output values from those nodes, before any further processing.

Why raw output can’t be used directly as probabilities

This is worth being precise about, because it explains exactly why an extra processing step is necessary. Logits are just ordinary numbers — they can be positive, negative, any size at all, and they don’t add up to any particular total. Real probabilities have to follow specific rules: every value has to be between 0 and 1, and the whole set has to add up to exactly 100%. A raw logit of, say, 4.7 doesn’t mean anything on its own — it’s just a relative score, meaningful only in comparison to the other tokens’ logits, not yet a usable probability.

flowchart LR
    A[Output Layer: one node per vocabulary token] --> B["Raw logits: 'blue' = 4.7, 'clear' = 3.2, 'falling' = -1.1, ..."]
    B --> C[Softmax conversion]
    C --> D["Actual probabilities: 'blue' = 45%, 'clear' = 20%, 'falling' = 2%, ..."]

ANALOGY vs. TECHNICAL REALITY

Analogy: Think of judges at a talent competition each assigning a raw numerical score to a performance — one judge gives a 9.2, another gives a 7.8, another gives a -2 (a judge who really disliked it, perhaps using a scale that allows negative scores). These raw scores tell you relative ranking — 9.2 is clearly the most impressed judge — but they don’t directly tell you “this performance had an 87% approval rating” until someone converts those raw scores into a standardized, comparable percentage.

Where this breaks down: Judges’ raw scores come from genuine subjective impression. A model’s logits come from the exact weighted-sum-plus-bias calculation covered in the Node article, computed by the output layer’s nodes — a purely mechanical result of the model’s learned parameters, with no subjective impression involved, just the accumulated result of everything computed in the layers before it.

Why logits are still genuinely useful on their own

It’s worth noting that logits aren’t just a throwaway intermediate step — their raw, relative values carry real, usable information even before conversion to probabilities. The relative ordering of logits is already meaningful: whichever token has the highest logit will also have the highest probability after conversion, since the conversion process, covered in the next article, preserves relative ranking. This is why some technical applications — like certain evaluation and research tools — work directly with logits rather than the converted probabilities, since the ordering and relative spread of raw logits can reveal useful information about how confidently or uncertainly a model is distinguishing between its top candidates.

A concrete example, layered

For a simple beginner example: given the sequence “The sky is”, a small model’s output layer might calculate raw logits of 5.1 for “blue”, 4.3 for “clear”, and -2.0 for “furniture” — numbers that already clearly rank “blue” as most likely and “furniture” as extremely unlikely, even before any further conversion happens. For a production example: a model like GPT-3, with a vocabulary of 50,257 tokens as confirmed in OpenAI’s published paper, calculates exactly 50,257 individual logit values at every single next-token prediction step — one raw score for every single possible token in its entire vocabulary, every single time it generates one more piece of text.

Calculate logits for a tiny vocabulary

Assume the final hidden vector is:

h = [0.5, 1.0]

Three output tokens have learned weight vectors and biases:

Token	Weight	Bias	Logit calculation	Logit
blue	[2, 1]	0	(0.5×2) + (1×1)	2.0
clear	[0, 1]	0	(0.5×0) + (1×1)	1.0
dark	[0, 0]	0	0	0.0

logits = [2.0, 1.0, 0.0]

The values need not be between 0 and 1 and do not sum to 1. Their relative sizes determine the softmax probabilities.

How GPT and Gemini use logits

At each text-generation step, GPT and Gemini-style models project the final hidden representation into one logit per possible output token. Sampling controls operate after or around this scoring stage.

OpenAI’s API documentation explains that logit_bias values are added to model logits before sampling in APIs that support that option. This can make specified token IDs more or less likely, demonstrating that logits are the raw control point before probability-based selection.

Common misconception

A frequent beginner assumption: that logits and probabilities are basically the same thing, just described with different words. As this article has explained, they’re genuinely different — logits are raw, unbounded numbers with no fixed total, while probabilities are the specific, standardized output of converting logits through the softmax function, always between 0 and 1 and always summing to exactly 100%. Confusing the two can lead to real misunderstandings — a logit of 4.7 is not “a 4.7% chance,” it’s simply a raw score that needs proper conversion before it means anything in probability terms.

Where this fits in what comes next

You now understand the raw, unconverted numbers a model calculates first. The next article, Probability Distribution, covers exactly what those logits get converted into — the proper, standardized set of probabilities across every possible next token that the model actually uses to make its final choice.

In one sentence

Logits are the raw, unnormalized scores a model’s output layer produces for every possible next token — real and relatively meaningful in their ordering, but not yet a genuine probability until they go through the conversion process covered in the very next article.

Related Terms

• Output Layer
• Node
• Next-Token Prediction
• Probability Distribution
• Prediction