Glossary
What is data normalization?
In short
Data normalization brings numeric features to a comparable scale, for example to the range from zero to one, so that none of them outweighs the others only because of its unit of measure.
Machine learning · normalization
MMOOZZGG.com editorsChecked: 2 min read
For a child
Vasya measured his friends’ height in centimeters, and Petya in meters. Vasya got big numbers like one hundred thirty, and Petya small ones like one point three. If you just add these records together, you get a mess. First you have to convert everything into the same units. Before it learns, a computer also brings numbers to one form, so that big ones do not hide small ones.
In plain words
In a customer table, age is measured in tens and income in hundreds of thousands. Many models take a number literally: the feature with more digits seems more important. To prevent this, values are recalculated. You can squeeze each column into the range from zero to one, where zero means the minimum and one the maximum. Or you can subtract the mean and divide by the spread, and then every column has a mean of zero. After this, age and income speak to the model on equal terms. It is like comparing prices in different currencies: first you convert everything into one currency, then you compare.
For business
The procedure itself costs almost nothing, but mistakes in it are expensive. A typical story: the scaling parameters were calculated on all the data at once, including future months, and in testing the model looks stronger than it will be in real work. Another: at launch, someone forgot to apply to new data the same scaling as in training, and the forecasts went wrong. Outliers, such as one deal worth a billion, squeeze the other values into a narrow band, so for such data teams use methods that resist outliers. Decision trees and gradient boosting do not need rescaling, but linear models with regularization and neural networks almost always do. At acceptance, ask the vendor to show that scaling is part of one processing pipeline and is stored with the model. A company’s data readiness for such models is assessed in advance, when preparing data for AI.
For a specialist
Data normalization (feature scaling, normalization) is the transformation of features to a common scale. Min-max normalization (MinMaxScaler) maps a feature linearly to a range, [0, 1] by default; standardization (z-score, StandardScaler) subtracts the mean and divides by the standard deviation, giving a zero mean and unit variance; MaxAbsScaler divides by the maximum absolute value and keeps sparsity; RobustScaler relies on outlier-resistant estimates of center and spread and suits data with outliers. Sample normalization (Normalizer) is a separate thing: each row is brought to a unit L1 or L2 norm, which helps for dot products and cosine similarity of text vectors. Scale matters for SVM with an RBF kernel, for linear models with L1 and L2 regularization and for gradient descent, where different scales stretch the loss surface and slow convergence. Statistics are computed on the training set only, and the transformation is built into a Pipeline to rule out leakage. Inside neural networks, batch normalization and layer normalization play a similar role.
Do not confuse. Not to be confused with database normalization, the term for splitting tables to remove duplication.
Example
A churn forecast model looks at customer tenure in days and the total of purchases in money. Before rescaling, it almost ignored tenure, because the amounts were orders of magnitude larger. After standardization, both features began to affect the forecast, and the model started to notice newcomers who leave after their first purchase.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map