MMOOZZGG.com

Glossary

What is underfitting?

In short

Underfitting is an error in which a model is too simple or trained too little, so it does not catch the pattern even in the data it learned from.

Machine learning · underfitting

MMOOZZGG.com editorsChecked: 2 min read

For a child

Imagine you draw a cat as one small circle. Such a circle does not look like a cat, a dog or a rabbit: it has too few details. A computer sometimes learns the same way and remembers a rule that is too rough. Then it gets confused both on familiar pictures and on new ones. This is called underfitting.

In plain words

A model looks in examples for a rule that turns input information into an answer. If the model is too simple or was stopped too early, the rule comes out rough. For example, someone tries to guess an apartment’s price only from its area, ignoring the district, the floor and the renovation. Such a program is wrong everywhere: on familiar examples and on new ones. This is how it differs from an overfitted model, which shines on familiar data. It helps to make the model more complex, give it more useful information or train it longer. It is like a student who read only the table of contents of a textbook and so answers any question poorly.

For business

An underfitted model looks honestly weak: the vendor shows modest accuracy on both training and validation. The danger is the hasty conclusion “AI does not suit us”, when the problem is in how the task was set. Common causes: the model was not given important data (seasonality, customer history, stock from 1C), the algorithm chosen was too simple, or the limits against overfitting were too strict. A fix is usually cheaper than a new project: add features, move to a more flexible model such as a random forest, or loosen the limits. Before deciding, compare the result with the simple rule the department uses now. If the model does not beat even that, first look for the missing data. A pilot is a good place for such a comparison.

For a specialist

Underfitting is a regime in which a model does not reach a low error even on the training set: its capacity or training time is not enough to describe the dependency in the data. In the bias-variance decomposition of error, this is high bias. The scikit-learn documentation states the sign through curves: if both the training and the validation scores are low, the estimator is underfitting; if both converge to a low value as the sample grows, new data is unlikely to help. Remedies: a more expressive model (polynomial features, deeper trees, more layers), feature engineering, weaker regularization or dropout, more epochs and a tuned learning rate. For diagnosis, teams plot learning curves by data volume and validation curves by one hyperparameter. The opposite regime with high variance is called overfitting, and tuning happens between these two poles.

Example

A bakery chain forecast bread demand with a linear model based only on the day of the week. The forecast missed equally badly on old weeks and on new ones. When weather and holidays were added to the model, the error fell on both training and validation.

Related terms

Questions and answers

How do you tell underfitting from overfitting? Compare the error on training data and on validation data. With underfitting, both are high and close to each other.

With overfitting, the training error is small, and the validation error is much larger.

Does more data help with underfitting? Usually not much. The model lacks flexibility or useful features, and new examples of the same kind will not fix that.

First make the model more complex or add the information it is missing.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us