Glossary
What is a baseline model
In short
A baseline model is a simple starting solution to a task. You compare a new model with it to see whether the new model gives a real gain.
Machine learning · baseline
MMOOZZGG.com editorsChecked: 1 min read
For a child
You are learning the long jump. On the first day you measure how far you jump with no training and write it in a notebook. A month later you jump again and see how much farther you go. Without that first note, you would not know whether training helped. Computer programs are also compared with such a first note, to see if they became smarter.
In plain words
Before you praise a model, you must know what to compare it with. This reference point is called the baseline. Usually it is a very simple rule: always give the most common answer, forecast sales as in the previous month, search by keywords. If a complex model guesses nine cases out of ten, it sounds good. But if nine emails out of ten are not spam, the rule “all emails are normal” gives the same result and costs nothing. A useful model must clearly beat such a reference point, or its complexity is not needed.
For business
Without a reference point you cannot honestly calculate the return on AI. A project usually has two. The technical one: how much more accurate the new model is than a simple rule or the current algorithm. The business one: how the process works now, how long an operation takes, what it costs and how often people make mistakes. Record the business baseline before the start, or a month later there will be nothing to compare with and the effect turns into a dispute of opinions. A common trick is to compare with a deliberately weak option so the result looks impressive. Ask the contractor to show which baseline it took and why. In a pilot with a metric in the contract, the starting figures of the process are recorded on the first day.
For a specialist
A baseline is a reference solution against which you measure the quality gain of a new model on the same sample, the same metric and the same validation scheme. Levels: a trivial (dummy) baseline that ignores the features; a heuristic one, such as an expert rule, a “same as yesterday” forecast or a seasonal naive forecast for time series; a simple trainable model such as logistic regression; the previous version of the system in production, or a published result on a benchmark. In scikit-learn the trivial options come from DummyClassifier with the strategies most_frequent, prior, stratified, uniform and constant, and from DummyRegressor for regression. The comparison is fair only if the reference goes through the same pipeline: the same split, the same data normalization and the same hyperparameter search budget. Check the gain statistically, for example with a bootstrap confidence interval. With class imbalance, look not at accuracy but at the F1 score and the confusion matrix.
Example
A contractor reports that the model correctly finds customers about to leave in nine cases out of ten. An analyst checks the simple rule “nobody will leave” and gets almost the same result, because few customers leave. After this comparison the model goes back for rework.
Related terms
Questions and answers
Sources
Where companies use it
Not sure whether your company needs this?
AI solutions map