The Scaling Laws article covered smooth, predictable improvement as models scale up. This article covers a related but stranger, more contested phenomenon: capabilities that seem to appear suddenly, not smoothly, called emergent abilities.
The simple definition
Emergent abilities are capabilities that appear to show up suddenly at a specific model scale, with a model performing at essentially random chance below that scale and then jumping to well-above-random performance above it — rather than improving gradually the whole way up, as scaling laws would predict for most metrics. Recall from the Large Language Model article’s early discussion of this exact phenomenon, first raised there and given its full, dedicated treatment here.
Why this looked like a genuinely different pattern from ordinary scaling
Recall from the Scaling Laws article’s description of smooth, predictable power-law curves. Most measured properties of language models — like the raw loss value covered throughout the Loss Function article — do follow this smooth pattern as models scale up. But a 2022 paper by Wei and colleagues documented something that looked different for certain specific task performance metrics: a model’s ability to do certain kinds of few-shot arithmetic, or multi-step reasoning, would sit at essentially random, chance-level performance across a wide range of smaller model sizes, then jump sharply to genuinely useful performance once the model crossed a certain scale threshold — no smooth, gradual improvement in between, just an apparent discontinuity.
flowchart LR
A[Small models: task performance stays near random chance] --> B[Scale threshold crossed]
B --> C[Larger models: performance jumps sharply above random]ANALOGY vs. TECHNICAL REALITY
Analogy: Think of water heating on a stove — its temperature rises smoothly and predictably, degree by degree, right up until it hits 100°C, at which point it suddenly transforms into steam, a qualitatively different state, not just “slightly hotter water.” Emergent abilities were described using exactly this kind of “phase transition” language — a model isn’t just gradually getting a little better at a task, it seems to cross a genuine threshold into being able to do something it fundamentally couldn’t do at all before.
Where this breaks down: Water boiling is governed by well-understood, settled physics with a precise, universally agreed transition point. Whether emergent abilities represent a genuine phase transition in a model’s underlying computation, or something else entirely, is precisely the contested scientific question the rest of this article covers — the analogy captures the visual, surprising shape of the phenomenon, not a settled explanation of its actual cause.
The real scientific fight: is this even real?
This is worth telling as a genuine, still-unresolved controversy, not a settled fact. In 2023, a paper by Schaeffer, Miranda, and Koyejo, provocatively titled “Are Emergent Abilities of Large Language Models a Mirage?”, directly challenged the original finding. Their core argument: the apparent sudden “jump” wasn’t necessarily a real change in the model’s underlying capability at all — it could instead be an artifact of how researchers chose to measure performance. Many of the tasks showing “emergent” jumps were scored using strict, all-or-nothing metrics (like requiring an exact match on a multi-step arithmetic answer) — and the researchers showed that switching to a smoother, more gradual scoring metric on the exact same models and tasks often made the sudden jump disappear entirely, revealing steady, gradual improvement underneath all along. In other words: the model might have been improving smoothly the whole time; it just looked sudden because of how success was being counted. This paper won a Outstanding Paper award at NeurIPS 2023, one of the field’s top research venues — a genuine, high-profile scientific rebuttal, not a minor footnote.
Why this debate genuinely matters, beyond academic interest
It’s worth being direct about the real stakes here. If emergent abilities are genuine, sudden phase transitions, this has serious implications for AI safety research, covered throughout the AI Safety article — a capability could appear unexpectedly at some future scale with no gradual warning sign beforehand. If emergent abilities are largely a measurement artifact, as Schaeffer and colleagues argued, then capability growth is more predictable and gradual than the original framing suggested, a meaningfully different, more reassuring picture for anticipating what a next-generation model might suddenly be capable of.
A concrete example, layered
For a simple beginner example: a model tested on three-digit addition might score essentially 0% correct across many smaller model sizes, then jump to 80% correct once it crosses a certain parameter threshold — the “sudden emergence” pattern Wei and colleagues documented. For a production example: this exact debate has real, direct relevance to how companies like OpenAI, Anthropic, and Google DeepMind think about capability forecasting — trying to anticipate what a next-generation model might suddenly be able to do is a genuinely different planning problem depending on whether new capabilities appear gradually (predictable, if you’re tracking the right smooth metric) or as true, sharp discontinuities (much harder to anticipate in advance).
Where the science currently stands
It’s worth being honest that this remains a genuinely open, active research area rather than a fully resolved dispute. Subsequent research has continued investigating both sides — some published work has found genuine, measurement-independent discontinuities in specific narrow cases, while other work has continued supporting the “mostly a metric artifact” explanation for many originally reported examples. The most balanced, current summary: at least some claimed emergent abilities appear to be measurement artifacts, as Schaeffer’s team demonstrated, while the underlying question of whether any genuine, measurement-independent discontinuities exist in language model capabilities remains a real, unsettled empirical question.
Why the graph can fool us
Suppose exact-match scoring gives one point only when every digit is correct:
Model A: 4 of 10 digits correct → exact-match score 0
Model B: 8 of 10 digits correct → exact-match score 0
Model C: 10 of 10 digits correct → exact-match score 1The underlying ability improved from 4 to 8 to 10 correct digits, but the graph jumps from 0 to 1 only at Model C. A smoother “fraction of digits correct” metric would display gradual progress. This is the central intuition behind the argument that some apparent emergence is caused by the evaluation metric.
The original Google-led paper defined an ability as emergent when it is absent in smaller models but present in larger ones. The later “mirage” paper tested InstructGPT/GPT-3 and BIG-Bench results and found that several sharp jumps became smooth under continuous metrics. Sources: Emergent Abilities of Large Language Models and Are Emergent Abilities of Large Language Models a Mirage?.
Common misconception
A frequent beginner assumption: that “emergent abilities” is a settled, well-established fact about how language models work, the way scaling laws largely are. As this article’s controversy section has explained, this overstates the current scientific consensus — the original 2022 finding was genuinely influential and widely cited, but it was directly and credibly challenged within a year, and the field has not settled definitively on one side of that debate.
Where this fits in what comes next
You now understand both the striking original finding and the genuine scientific pushback it received. The next article, Distillation, shifts to a much more settled, practical technique — compressing a large, capable model’s knowledge into a smaller, faster one, with its own real, quantified successes and a genuine, recent controversy of its own.
In one sentence
Emergent abilities describe capabilities that appear to jump suddenly at certain model scales, a striking 2022 finding that was directly challenged by a credible, award-winning 2023 paper arguing much of the apparent suddenness was a measurement artifact — a genuine, still-unresolved scientific debate rather than settled fact, with real consequences for how the field thinks about anticipating future model capabilities.
Related Terms
- Author
- TechByteByByte Editorial Team
- Reviewed by
- TechByteByByte Admin
- Published
- Last reviewed