MMOOZZGG.com

Glossary

What is cross validation

In short

Cross validation is a way to evaluate a model in which the data is split into parts that take turns as training and test sets, and the results are averaged.

Machine learning · cross-validation

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine there are five teams in your class and you hold a quiz. Each time, one team makes up the questions and the others answer. Then the roles change, until every team has made up questions. This shows who really knows the subject and who just got easy questions. A computer is tested in a similar way, in turns and many times.

In plain words

If you set aside one part of the data for testing, the result can depend on luck: you may get easy examples, and the model looks stronger than it is. So the data is split, for example, into five equal parts. The model trains on four, is tested on the fifth, and this repeats until each part has served as the test set. The five scores are averaged, and you look at how widely they spread. This way every example is used, and the conclusion is more stable. It is like a tasting where each dish is tried in turn by different guests, not by one.

For business

You need this method when data is scarce: hundreds of customers, dozens of equipment failures, rare fraud cases. On such volumes one random split can give either delight or disappointment. The downside is cost: the model trains as many times as there are parts, so computing grows five to ten times. For large neural networks this is often too much, and teams make do with one test part. The trap is data over time: you must not mix sales from different years, or the model “peeks” at the future and the score is inflated. Ask how the data was split and how widely results spread between parts. A wide spread means it is too early to trust the promised number. This matters most in forecasts of demand and load.

For a specialist

Cross validation is a family of methods that estimate how well a model generalizes by splitting the data repeatedly. In the k-fold scheme, the sample is split into k parts; the model trains k times on k-1 folds and is tested on the remaining one, and the metric is averaged. According to the scikit-learn documentation, most authors and empirical evidence favor 5 or 10 folds over leave-one-out, which gives a high-variance estimate. Variants: stratified (StratifiedKFold) keeps class shares in the folds; grouped (GroupKFold) does not let one object, such as a patient, appear in both training and testing; for time series, TimeSeriesSplit trains on the past and tests on the next stretch, because the usual schemes assume independent, identically distributed examples. Preprocessing is fitted inside each fold through a Pipeline. To tune hyperparameters and still estimate the result honestly, teams use nested cross validation.

Example

A manufacturer had only eighty pump failures in five years. Instead of one split, the team ran five-fold cross validation grouped by pump and saw that accuracy jumped a lot from fold to fold. It became clear there was not yet enough data for launch.

Related terms

Questions and answers

Why do you need cross validation? It gives a more reliable estimate of a model on small data, because every example lands in the test set at least once. It also shows how much the result depends on a random split.
How many folds should you choose for cross validation? Usually five or ten: a balance between the accuracy of the estimate and the computing time. For time series, the folds follow the order of dates, not a random order.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us