Imagine judging two athletes by comparing their raw stats: one runs 100 meters in 11 seconds, the other lifts 150 kilograms. You can’t meaningfully compare “11” and “150” directly — they’re measuring completely different things, on completely different scales. Before you could combine these into any kind of fair overall score, you’d need to put them on some kind of comparable footing first. That exact problem, applied to a model’s Features, is what normalization solves.
The simple definition
Normalization is the process of rescaling numeric features so they sit on a similar, comparable range, before a model trains on them. It’s one of the specific preprocessing techniques introduced in the previous article, Data Preprocessing — but important and common enough to deserve its own close look.
Putting different scales on comparable ranges
Consider two house features:
Floor area: 1,200 square feet
Bedrooms: 3The area number is much larger, but that does not automatically mean it should dominate learning. Normalization rescales numerical values while preserving their useful relationship.
Min-max normalization with numbers
Suppose training floor areas range from 500 to 2,000 square feet. For a 1,200 square-foot house:
normalized value = (value - minimum) / (maximum - minimum)
= (1,200 - 500) / (2,000 - 500)
= 700 / 1,500
= 0.467The original value becomes approximately 0.467 on a scale where the training minimum is 0 and maximum is 1.
Standardization with numbers
Suppose the training mean is 1,000 square feet and the standard deviation is 250:
standardized value = (value - mean) / standard deviation
= (1,200 - 1,000) / 250
= 0.8The result means the house is 0.8 standard deviations above the training mean.
Avoiding leakage
Calculate the minimum, maximum, mean, or standard deviation using training data only. Store those values and reuse them.
Training data → calculate scaling values → store scaler
Validation data ─┐
Test data ───────┼→ use stored scaler
Production input ┘A code example
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train) # Learn mean and standard deviation from training data
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)When normalization matters
Distance-based models, gradient-based models, and neural networks often benefit from scaling. Many tree-based models are much less sensitive because they split values by thresholds rather than comparing raw magnitudes directly.
Why unscaled features actually break things
This isn’t a cosmetic concern — it’s a real, technical problem with real consequences for training. Recall from the Training article that a model learns by adjusting its internal parameters based on gradient descent, nudging values in the direction that reduces error.
If one feature — say, a house’s square footage, ranging from about 500 to 5,000 — sits on a wildly different numeric scale than another feature — say, number of bedrooms, ranging from 1 to 6 — the training process can behave badly.
The feature with the larger raw numbers can end up dominating the math simply because its numbers are bigger, not because it’s actually more important to the prediction, and training can become slow, unstable, or get stuck in a way that never properly balances the different features’ real influence.
flowchart LR
A[Square footage: 500-5000] --> C[Without normalization: dominates training]
B[Bedrooms: 1-6] --> D[Without normalization: barely influences training]
A --> E[Normalization: both rescaled to 0-1]
B --> E
E --> F[Balanced, stable training]ANALOGY vs. TECHNICAL REALITY
Analogy: Think of grading a job candidate on a 0–10 interview score and a 150,000 salary history, then trying to average the two into one combined number. The salary figure would completely swamp the interview score just because its numbers are so much bigger — even if, in reality, you wanted both to matter roughly equally.
Where this breaks down: A hiring manager, noticing this problem, can consciously decide to weigh the two fairly. A model doesn’t notice anything — it mechanically processes whatever numbers it’s given, exactly as they are, with no built-in sense that “150,000” and “8” are meant to represent comparably important pieces of information. Normalization is the deliberate, upfront step that prevents this imbalance before training ever begins, since the model itself has no way to correct for it on its own.
The two most common normalization techniques
There isn’t just one way to rescale numbers, and choosing the right one depends on the data and the situation:
- Min-max scaling rescales every value in a feature to fall between a fixed range, usually 0 and 1, based on that feature’s actual minimum and maximum. A house’s square footage of 2,000, in a dataset ranging from 500 to 5,000, might become roughly 0.33. This is intuitive and works well when you know the feature has a reasonably stable, bounded range.
- Standardization (sometimes called z-score normalization) rescales values based on the feature’s average and how spread out its values typically are, so that the resulting numbers represent how many standard deviations above or below average a given value is. This tends to handle outliers a bit more gracefully than min-max scaling, and is a very common default choice in practice.
Both techniques accomplish the same underlying goal — putting features on a comparable footing — through slightly different math, and an engineer chooses between them based on the specific characteristics of the dataset at hand.
A concrete example, layered
For the house price example used throughout this glossary: without normalization, square footage (in the thousands) could end up overwhelming the model’s attention compared to number of bedrooms (single digits) or age of the house in years (double digits), even if bedrooms genuinely matter a great deal to price. After normalization, all three features sit on comparable numeric footing, and the model’s training process can weigh their actual predictive importance fairly, rather than being distorted by arbitrary differences in raw units.
Where this genuinely matters most — and where it barely matters at all
Normalization matters enormously for algorithms that are directly sensitive to the raw scale of numbers — including the gradient-descent-based training that underlies the Neural Networks and Deep Learning systems behind today’s large language models, where unnormalized inputs can meaningfully slow down or destabilize training at scale. It also matters for distance-based algorithms, which compare how “close” data points are to each other numerically. It matters much less for certain algorithms, like decision trees, which make decisions based on comparisons and thresholds rather than raw numeric distances, and are largely unaffected by the original scale of a feature.
This is an important nuance: normalization isn’t a universal requirement for every ML technique — it’s specifically important for the algorithm families where scale genuinely affects the math, and an engineer’s choice of algorithm partly determines how much this step matters for a given project.
Key terms
- Scale: The numerical range of a feature.
- Min-max normalization: Rescaling using training minimum and maximum.
- Standardization: Centering around the mean and scaling by standard deviation.
- Scaler: Stored transformation values and logic.
- Outlier: An unusually extreme value that can affect scaling.
Check your understanding
Does normalization delete the original ordering? No. Larger values remain larger under common monotonic scaling methods.
Should test data determine the scaler? No. That creates information leakage.
Common misconception
People sometimes assume normalization changes what a feature actually means — that scaling square footage down to a number between 0 and 1 somehow loses real information. It doesn’t. Normalization is a purely mathematical rescaling; the relative relationships between values are preserved exactly (a house that was twice as large as another before normalization is still, proportionally, roughly twice as large after). What changes is only the numeric range the values sit in, not the actual pattern or relationship the data represents.
Another common mix-up: normalizing the training, validation, and test sets separately, using each one’s own minimum and maximum. This subtly leaks information about the test set into the process and can distort results. The correct practice is to calculate the normalization parameters (like min/max or average/spread) using only the training data, and then apply that exact same rescaling to the validation and test sets — keeping the strict separation established back in the Train-Test Split article intact.
Normalization and standardization are related, not identical
Some people use normalization as a broad name for rescaling. More precisely, min-max normalization usually maps values into a chosen range, while standardization describes values by their distance from the mean in standard-deviation units.
The minimum, maximum, mean, and standard deviation must be learned from training data only. Save those learned statistics with the preprocessing pipeline, then reuse exactly the same values for validation, test, and live inputs.
Where this fits in what comes next
Normalization handles how existing numeric features are scaled. The next article, Feature Engineering, goes a step further — not just rescaling what’s already there, but actively creating new, more useful features from the raw, cleaned data, a topic first introduced briefly back in the Feature article and given its full, dedicated treatment next.
In one sentence
Normalization puts a dataset’s numeric features onto a comparable scale so that a model’s training process weighs each feature by its actual predictive importance, not by the arbitrary size of the units it happens to be measured in.
Related Terms
- Author
- TechByteByByte Editorial Team
- Reviewed by
- TechByteByByte Admin
- Published
- Last reviewed