Batch

The Gradient Descent article mentioned, in passing, that calculating a gradient using an entire training dataset before taking even one step is impractically slow — and that real training instead uses small groups of examples at a time. This article gives that idea its proper name: a batch.

The simple definition

A batch is a small group of training examples processed together before the model’s weights get updated once. Rather than calculating a gradient using all of Training Data at once, or using just one example at a time, training typically splits the training set into many smaller batches, and performs one gradient descent update after processing each batch. Feed the model one batch, calculate the loss and gradient across just those examples, update the weights, then move on to the next batch — repeating this all the way through the training set.

Dividing 1,000 examples into groups

Suppose training data contains 1,000 examples and batch size is 100:

Batch 1: examples   1–100
Batch 2: examples 101–200
...
Batch 10: examples 901–1,000

The model processes one batch, calculates an average loss and gradients, and performs an update before moving to the next batch.

flowchart LR
    A[Load one batch] --> B[Forward pass]
    B --> C[Average batch loss]
    C --> D[Backpropagation]
    D --> E[Parameter update]
    E --> F[Load next batch]

After all ten batches have been processed once, the model has completed one epoch.

Why not use all examples at once?

The full dataset may not fit in GPU memory. Smaller groups also allow the model to update more frequently. However, each batch provides only an estimate of the gradient for the whole dataset, so updates contain some noise.

Why batches exist at all

This solves a genuine, practical problem raised at the end of the Gradient Descent article. Calculating a gradient using the entire dataset before every single update — sometimes called batch gradient descent in its original, full form — means a model trained on millions of examples would need to process every single one of them just to take one small step.

For a dataset the size of the ones described in the Training Data article, that would mean waiting an enormous amount of time between each tiny weight adjustment, making the whole process painfully inefficient. Batches solve this by letting the model take many more update steps in the same amount of time, using a smaller, faster-to-process slice of the data for each one.

flowchart LR
    A[Full Training Dataset] --> B[Split into many batches]
    B --> C[Batch 1: calculate gradient, update weights]
    C --> D[Batch 2: calculate gradient, update weights]
    D --> E[...continues through all batches]
    E --> F[One full pass = one Epoch]

ANALOGY vs. TECHNICAL REALITY

Analogy: Think of a teacher grading a massive stack of exams and wanting to adjust their teaching approach based on common mistakes. Rather than waiting to grade all 500 exams before making any adjustment, they grade a stack of 20, notice a pattern, adjust their next lesson slightly, grade the next 20, adjust again — reacting and improving continuously throughout, rather than waiting for one enormous, delayed judgment at the very end.

Where this breaks down: A teacher consciously interprets patterns in each stack of 20 exams. A batch’s contribution to a weight update is a purely mechanical average of the gradients calculated across its examples, with no interpretation involved — and unlike a teacher’s evolving judgment, the model has no memory of “this pattern showed up in the last batch” beyond what got encoded into its weights at that step.

What using a batch (instead of the full dataset) actually costs

This is a genuine trade-off worth being honest about, not a free efficiency win. A gradient calculated from just one batch is a noisier, less precise estimate of what the true gradient across the entire dataset would say — since a small slice of data doesn’t perfectly represent the whole.

This means each individual weight update, based on a single batch, is a bit rougher and less exact than an update based on the full dataset would be.

In practice, this noisiness turns out to be a reasonable, even beneficial trade — the model takes far more update steps overall, and the added noise sometimes even helps training escape a poor local minimum (the concern raised in the Gradient article) that a smoother, more “confident” full-dataset gradient might have settled into and stayed stuck in.

Where a batch fits into the full training cycle

A batch is the unit of work between weight updates; an Epoch, covered shortly in this phase, is one complete pass through every batch in the training set. If a training set has 10,000 examples, split into batches of 100 examples each, one epoch consists of 100 separate batches, meaning 100 separate weight updates happen in the course of a single full pass through the data — and training typically repeats this process across many epochs, as covered in the Training article.

A concrete example, layered

For a simple beginner example: training the one-weight house model on a dataset of 1,000 houses, split into batches of 50, means the model updates its weight 20 separate times over the course of one full pass through the data — 20 small, incremental corrections instead of one single, delayed correction based on all 1,000 houses at once.

For a production example: OpenAI’s published GPT-3 training recipe processed its roughly 300-billion-token training set using a batch size of 256 sequences — meaning the model’s 175 billion parameters were updated once per 256-sequence batch, continuously, across the entire multi-week training run, rather than waiting to process the full 300-billion-token dataset before a single update.

Check your understanding

Is a batch a permanent subdivision? Not necessarily. Data is often shuffled and grouped differently in later epochs.

Does one batch equal one epoch? Only when the batch contains the entire training dataset.

Common misconception

A frequent early mix-up: assuming a batch is just an arbitrary technical detail, unrelated to a model’s final quality — purely a speed optimization with no real consequence beyond how fast training runs. In practice, as the trade-off section above explained, the size of a batch genuinely affects training dynamics, not just speed — it influences how noisy or smooth each gradient estimate is, which in turn affects how training actually behaves.

This is exactly why the next article in this phase, Batch Size, deserves its own full treatment rather than being a footnote here: the specific number of examples per batch is itself an important, tunable hyperparameter, not a fixed technical constant.

Batch and mini-batch

Full-batch training uses the whole dataset for one update. A mini-batch uses only a manageable subset. In modern deep-learning conversations, people usually say “batch” when they mean “mini-batch.”

If 1,000 examples are divided into batches of 50, one epoch contains 20 batches and normally 20 optimizer updates. The final batch can be smaller when the dataset size is not evenly divisible.

Where this fits in what comes next

You now understand what a batch is: the group of examples a model processes together between weight updates. The next article, Batch Size, covers the practical question this article has deliberately left open — exactly how many examples should go into each batch, and what happens when that number is set too small or too large.

In one sentence

A batch is the small group of training examples a model processes together before taking one gradient descent step, and it’s the practical mechanism that makes training on massive, real-world datasets computationally feasible at all.

Related Terms

• Gradient Descent
• Training
• Batch Size
• Epoch
• Training Data