MMOOZZGG.com

Glossary

What is a baseline model

In short

A baseline model is a simple starting solution to a task. You compare a new model with it to see whether the new model gives a real gain.

Machine learning · baseline

MMOOZZGG.com editorsChecked: 1 min read

For a child

You are learning the long jump. On the first day you measure how far you jump with no training and write it in a notebook. A month later you jump again and see how much farther you go. Without that first note, you would not know whether training helped. Computer programs are also compared with such a first note, to see if they became smarter.

In plain words

Before you praise a model, you must know what to compare it with. This reference point is called the baseline. Usually it is a very simple rule: always give the most common answer, forecast sales as in the previous month, search by keywords. If a complex model guesses nine cases out of ten, it sounds good. But if nine emails out of ten are not spam, the rule “all emails are normal” gives the same result and costs nothing. A useful model must clearly beat such a reference point, or its complexity is not needed.

For business

Without a reference point you cannot honestly calculate the return on AI. A project usually has two. The technical one: how much more accurate the new model is than a simple rule or the current algorithm. The business one: how the process works now, how long an operation takes, what it costs and how often people make mistakes. Record the business baseline before the start, or a month later there will be nothing to compare with and the effect turns into a dispute of opinions. A common trick is to compare with a deliberately weak option so the result looks impressive. Ask the contractor to show which baseline it took and why. In a pilot with a metric in the contract, the starting figures of the process are recorded on the first day.

For a specialist

A baseline is a reference solution against which you measure the quality gain of a new model on the same sample, the same metric and the same validation scheme. Levels: a trivial (dummy) baseline that ignores the features; a heuristic one, such as an expert rule, a “same as yesterday” forecast or a seasonal naive forecast for time series; a simple trainable model such as logistic regression; the previous version of the system in production, or a published result on a benchmark. In scikit-learn the trivial options come from DummyClassifier with the strategies most_frequent, prior, stratified, uniform and constant, and from DummyRegressor for regression. The comparison is fair only if the reference goes through the same pipeline: the same split, the same data normalization and the same hyperparameter search budget. Check the gain statistically, for example with a bootstrap confidence interval. With class imbalance, look not at accuracy but at the F1 score and the confusion matrix.

Example

A contractor reports that the model correctly finds customers about to leave in nine cases out of ten. An analyst checks the simple rule “nobody will leave” and gets almost the same result, because few customers leave. After this comparison the model goes back for rework.

Related terms

Questions and answers

What does baseline mean in machine learning? It is a simple solution you compare a new model with: a rule, the previous version of the system or a forecast based on the average. If the model does not clearly beat it, its complexity is not justified.
What other words mean baseline? People most often say base line, base level or reference point. Among Russian specialists the English word itself has also taken root.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us