Before you continue: three tools for this module
- Token: a piece of text processed by the model.
- Parameter: a learned number controlling the model’s transformations.
- Inference: using the trained model without updating its parameters.
You do not need to memorize these yet. Use this map when the terms reappear.
Begin with the central question
How can a computer learn to work with language and generate useful text without storing a ready-made answer for every possible question?
Keep that central question about What Is an LLM? in mind. The definitions, numbers, diagrams, and code examples below answer it one piece at a time.
text examples → training → learned language model → prompt → generated text1. What You Will Learn
Learning outcomes
- Define an LLM mechanically rather than through a marketing slogan.
- Explain what makes the model “large” and what its parameters contain.
- Trace how repeated next-token prediction produces longer text.
- Distinguish an LLM from a database, search engine, and complete AI application.
In one sentence
💡 Big picture
An LLM is a computer model that reads text as small pieces and repeatedly guesses which piece should come next.
2. Why This Module Exists
The problem this module solves
- People often imagine an LLM as a giant fact book, but it does not look up every answer from a hidden table.
- Understanding its next-token job makes later topics—tokens, training, prompting, and hallucinations—much easier to understand.
3. Intuition
an LLM is a function that takes in a sequence of tokens and outputs exactly one thing — a probability distribution over “what token comes next.” Everything else — chatting, writing code, summarizing — is this one operation, repeated, dressed up in a conversational interface.
Analogy: The Phone Keyboard Word-Predicter (Not a Lookup Index) Think of an LLM as an incredibly advanced version of your phone’s predictive text keyboard:
- The Wrong View (The Database): Many imagine the model holds a digital encyclopedia inside its files. When you ask “The capital of France is”, it looks up “France” in a capital-city table and fetches “Paris”.
- The Correct View (The Probabilistic Keyboard): The model contains no tables or search indices. It has billions of weight parameters representing language patterns.
- When you type “The capital of France is”, the model’s output layer triggers a voting contest across all 50,000 words. “Paris” receives 99% of the votes because the model has seen this specific context prefix millions of times during pretraining. It is completing a puzzle piece, not reading a database shelf.
See the complete journey inside an LLM
The diagram below connects the whole idea in one view. Follow the arrows from left to right: text becomes tokens, tokens become numerical vectors, Transformer layers process those vectors, and the model produces probabilities for the next token.
How to read this diagram accurately
- The shown token pieces, vector values, and probability percentages are simplified examples. A real tokenizer may split the same sentence differently, and a real model will calculate different numbers.
- Positional information is not usually passed forward as an independent word-like object. It is combined with each token’s embedding so the model can distinguish both meaning and order.
- The Transformer box shows the two central operations—self-attention and a feed-forward network—but a production Transformer layer also includes operations such as residual connections and normalization.
- The model does not always select the token with the highest probability. Some applications use sampling, which can choose another likely token to make the response less repetitive or more varied.
input text
↓
tokens → embeddings + position
↓
many Transformer layers
↓
next-token probabilities
↓
select one token → append it → repeat📊 Visual Flowchart: The Elements and Scaling of an LLM
Here is how compute, parameter sizing, and text corpora scale into emergent LLM capabilities:
graph TD
classDef corpus fill:#3498db,stroke:#333,stroke-width:1px,color:#fff;
classDef scale fill:#2ecc71,stroke:#333,stroke-width:1px,color:#fff;
classDef output fill:#f1c40f,stroke:#333,stroke-width:1px,color:#fff;
Data["Trillions of text tokens<br>(Web, Books, Code)"] --> Pretrain["1. Pretraining Phase"]:::scale
Compute["Thousands of GPUs/TPUs<br>(ExaFLOP budgets)"] --> Pretrain
Pretrain --> Weights["2. Parameter Weights Matrix<br>(Billions of learned floats)"]:::corpus
Weights --> DecoderStack["3. Decoder-only Transformer Stack<br>(Self-Attention + LayerNorm + FFN)"]:::scale
DecoderStack --> OutputHead["4. Output Projection LM Head"]
OutputHead --> ProbDist["5. Next Token Probability Distribution"]:::output
ProbDist --> Select["6. Pick token, append, repeat"]4. What Is a Large Language Model, Precisely?
Language Model: a model that assigns a PROBABILITY to a
sequence of text -- or, equivalently, predicts
the most likely NEXT token given what came
before
Large Language a language model built on the Transformer
Model (LLM): architecture (Transformers course), trained
on massive amounts of text, with a very large
number of parameters -- typically billionsWhy “large” specifically?
Large DATA: trained on hundreds of billions to trillions
of tokens of text
Large MODEL: billions to hundreds of billions of learned
parameters (Module 12)
Large COMPUTE: trained using enormous amounts of GPU/TPU
compute, over weeks to months (Module 13)None of these three “large” dimensions alone defines an LLM — it’s the combination, and specifically the emergent capabilities (Module 13) that appear once all three cross certain thresholds together.
5. What Problem Does It Solve? What Came Before?
You already traced this story completely in the NLP course:
Rule-based NLP (hand-written grammar rules)
↓
Statistical LMs (n-gram probability tables)
↓
RNNs (sequential hidden state)
↓
LSTMs (gated memory)
↓
Attention (direct, weighted relevance)
↓
Transformers (fully attention-based,
parallelizable architecture)
↓
Large Language Models (Transformers, scaled up
dramatically in data, params,
and compute)What makes an LLM different from earlier language models isn’t a new core mechanism — it’s the decoder-only Transformer architecture (Module 11) trained at a scale where genuinely new capabilities emerge (Module 13) that smaller models of the same architecture simply don’t exhibit.
6. How It Works — The One-Sentence Version
Given everything so far, predict a probability distribution over
what token comes next. Select one. Append it. Repeat.This is genuinely the entire mechanism — Module 5 covers it precisely, Module 7 traces the complete inference loop, and Module 10 shows exactly how the Transformer architecture you already know produces this distribution.
7. What Does the Data Actually Look Like?
At every single step, an LLM’s raw output is a vector of numbers — one number (“logit,” Module 5) per vocabulary token — converted via softmax into a genuine probability distribution that sums to exactly 1. Nothing more mysterious happens underneath.
8. Concrete Example — “The capital of France is ___”
# Build a small, inspectable example of What Is an LLM.
# Follow the inputs, transformations, and output in order.
import numpy as np
def softmax(x):
exp_x = np.exp(x - np.max(x))
return exp_x / np.sum(exp_x)
# Illustrative: what a TRAINED LLM's output layer might produce for
# "The capital of France is ___" -- these logits are hand-set to
# represent a plausible, ALREADY-TRAINED model's output (the full
# mechanism producing real logits is covered in Modules 5-10).
vocab = ["Paris", "London", "a", "the", "city", "Berlin", "France", "large"]
logits = np.array([8.2, 2.1, 1.5, 1.2, 0.9, 1.8, 0.7, 0.5])
probs = softmax(logits)
print("Prompt: 'The capital of France is ___'\n")
print("Token probabilities:")
for word, p in sorted(zip(vocab, probs), key=lambda x: -x[1]):
print(f" {word:8s}: {p:.4f} ({p*100:.1f}%)")
print(f"\nSum of all probabilities: {probs.sum():.4f}")
print(f"\nMost likely next token: '{vocab[np.argmax(probs)]}'")Expected Output:
Prompt: 'The capital of France is ___'
Token probabilities:
Paris : 0.9923 (99.2%)
London : 0.0022 (0.2%)
Berlin : 0.0016 (0.2%)
a : 0.0012 (0.1%)
the : 0.0009 (0.1%)
city : 0.0007 (0.1%)
France : 0.0005 (0.1%)
large : 0.0004 (0.0%)
Sum of all probabilities: 1.0000
Most likely next token: 'Paris'What this actually shows: the model did not look up “Paris” in a fact table. It produced a probability distribution over its entire vocabulary — every token got some probability, summing to exactly 1.0 — and “Paris” simply has the highest learned probability given this specific context.
This distinction matters enormously for understanding hallucination later (Module 21): the model is always doing this same probabilistic operation, whether the answer is a well-known fact or something it has no reliable basis for.
9. Where Transformers Fit Inside an LLM
LLM = decoder-only Transformer (Module 11)
+ massive pretraining data (Module 8)
+ massive parameter count (Module 12)
+ massive compute (Module 13)
+ (usually) instruction tuning + alignment (Modules 17-19)You already know the Transformer block completely — multi-head attention, residual connections, LayerNorm, the feed-forward network, stacked into a deep architecture. An LLM is this architecture; there is no separate “LLM mechanism” layered on top of it architecturally. What differs is scale and training procedure, not the core computation.
10. Language Model vs. LLM — Precisely
| Language Model (general) | LLM (specific) |
|---|---|
| Any model assigning probabilities to token sequences | A language model built specifically on Transformers, at large scale |
| Could be an n-gram model, an RNN, anything | Specifically decoder-only Transformer-based (Module 11), trained on massive data/compute |
Traditional NLP Models vs. Modern LLMs
| Traditional NLP (TF-IDF, classical ML, small RNNs) | Modern LLMs |
|---|---|
| Task-specific — one model per task (sentiment, NER, etc.) | General-purpose — one model, prompted differently, handles many tasks |
| Small, fast, cheap, interpretable | Large, capable of broad generalization, but expensive |
| Requires labeled data per task | Requires no task-specific labels for many uses (zero/few-shot via prompting) |
11. How Is This Used in Modern AI?
Trace it through a real model call
user message → assembled context → LLM computation → decoded output → application checksThis topic affects one stage of that path; it is not the complete product. Hosted GPT- and Gemini-style applications also add instructions, safety systems, retrieval, tools, serving infrastructure, and evaluation around the model.
🤖 How Is This Used in Modern AI?
Every product you interact with that “understands” or “writes” text — chat assistants, coding tools, search summarization — is, underneath, exactly this mechanism: a decoder-only Transformer repeatedly predicting a next-token probability distribution, selecting a token, and repeating.
12. How Is This Used in Agentic AI?
Separate the model from the runtime
goal + state + tool results → LLM proposal → runtime validation → execution or responseThe LLM proposes text or a structured action. Ordinary application code controls permissions, tools, retries, memory, and execution.
Direct relevance to Agentic AI: Foundational — this entire course exists because of it. Every agent’s “reasoning,” tool selection, and response generation is this exact mechanism, run repeatedly, with the agent’s context (system prompt, conversation, tool results) as the growing input sequence at each step.
13. Common Beginner Mistakes
⚠️ Mistake
Incorrect idea: assuming “large” refers only to parameter count.
Why it is incorrect: As shown directly, “large” spans data, parameters, AND compute together — and it’s specifically the combination, crossing certain thresholds, that produces the emergent capabilities distinguishing LLMs from smaller Transformer-based language models (Module 13).
⚠️ Mistake
Incorrect idea: believing the model retrieves facts from storage.
Why it is incorrect: As demonstrated directly, the model produces a probability distribution over its entire vocabulary — there’s no explicit “lookup” step anywhere in this mechanism, even when the output happens to be a correct, well-known fact.
⚠️ Mistake
Incorrect idea: thinking LLMs use a fundamentally different architecture from what you already learned in the Transformers course.
Why it is incorrect: They don’t — an LLM is a decoder-only Transformer (Module 11), trained at scale. Nothing architecturally new is introduced by “LLM” as a category.
14. Important Distinctions
| Language Model | LLM |
|---|---|
| General category | Specifically Transformer-based, at large scale |
| Traditional NLP | LLM |
|---|---|
| Task-specific models | General-purpose, prompted for many tasks |
| Base LLM | Instruction-Tuned / Aligned LLM (Modules 17-19) |
|---|---|
| Pure next-token prediction on raw text | Fine-tuned to follow instructions and align with human preferences |
15. What You Should Remember
- An LLM is fundamentally a decoder-only Transformer (architecture you already know completely), trained at large scale across data, parameters, and compute together.
- Its core operation is always the same: predict a probability distribution over the next token, select one, repeat.
- It does not look up answers — verified directly: even a correct, well-known fact emerges from a full probability distribution over the entire vocabulary, not a retrieval step.
16. Interview Questions
Beginner
Q: What is a Large Language Model?
Ans: A language model — a model that predicts the probability of the next token given prior context — built specifically on the Transformer architecture and trained at large scale across data, parameters, and compute, typically reaching billions of parameters.
Intermediate
Q: What’s the actual architectural difference between an LLM and the Transformer architecture you learned in the Transformers course?
Ans: There isn’t a separate architecture — an LLM is a decoder-only Transformer, the exact same block structure (multi-head attention, residuals, LayerNorm, feed-forward network) covered in full in the Transformers course, just trained at a much larger scale on much more data with much more compute.
“LLM” describes a scale and training regime, not a new architectural mechanism.
Advanced
Q: Why does an LLM producing “Paris” as the answer to “the capital of France is” not constitute the model “knowing” a fact in the way a database does?
Ans: As demonstrated directly, the model’s actual output at that step is a full probability distribution over its entire vocabulary, produced by the same next-token prediction mechanism regardless of whether the underlying content is a well-established fact or something more uncertain.
“Paris” wins simply because training pushed its learned probability higher than every alternative, given this specific context — there’s no separate factual-lookup mechanism distinguishing this case from any other prediction. This distinction becomes critical when reasoning about hallucination (Module 21): the mechanism producing a correct answer and an incorrect one is identical.
Scenario
Q: A colleague says “LLMs are just really big search engines.” How would you correct this, using what you’ve learned?
Ans: I’d point out that a search engine retrieves and ranks existing documents based on a query — its output is a set of pointers to existing content.
An LLM instead generates a probability distribution over its vocabulary at every step and produces new text, token by token, through this repeated prediction process — it has no mechanism for retrieving or citing specific source documents unless explicitly combined with retrieval (RAG).
The confusion is understandable since LLMs often produce factually accurate output, but the underlying mechanism (probabilistic next-token generation vs. document retrieval) is genuinely different.
AI Engineering
Q: Why does understanding “LLM = decoder-only Transformer at scale” matter practically, rather than just being a definitional nuance?
Ans: It directly explains what’s actually adjustable and what isn’t when working with LLMs in practice: the underlying architecture is fixed once trained (same as any Transformer, per the Transformers course), but scale-dependent capabilities (Module 13) mean model size selection is a genuine, practical trade-off — bigger models generally handle more complex tasks but cost more and run slower.
This framing also correctly sets expectations: prompting and fine-tuning (Modules 16-20) work within this fixed architecture, not by changing the underlying mechanism.
17. Next Step
Next: Module 2 — Tokens and Tokenization — the first stage of the pipeline this course traces end to end: how raw text becomes the token IDs an LLM actually processes.
- Author
- TechByteByByte Editorial Team
- Reviewed by
- TechByteByByte Admin
- Published
- Last reviewed