Glossary
What is the F1 score?
In short
The F1 score is one number from zero to one that joins the precision and the recall of a model, and it is high only when both are high.
Machine learning · F1 score
MMOOZZGG.com editorsChecked: 2 min read
For a child
Your teacher checks how well you found all the mistakes in a text. You could underline almost every word, so no mistake hides, but there will be a lot of extra lines. You could underline one word you are sure of, so nothing is extra, but almost everything is missed. You get a good grade only if you found almost all the mistakes and underlined almost nothing extra. A computer gets the same kind of overall grade.
In plain words
A model that searches for something makes two kinds of misses: it finds extra things and it misses needed ones. Precision shows what share of what it found is correct, and recall shows what share of what was needed it managed to find. Comparing models by two numbers is awkward, so they are joined into one. A plain average misleads here: a model that calls everything suspicious gets a recall of one and looks decent. So the harmonic mean is used, and it pulls toward the smaller of the two numbers. A high result is possible only if the model is both careful and attentive. Like an exam in two parts, where the weak part pulls the total down.
For business
This measure is handy when one number must describe how well a model finds rare but important cases: fraudulent payments, claims with a threat of court, defects on a line, requests that must not be lost. The share of correct answers misleads here, and the F1 score does not. But it has a weakness: a miss and a false alarm count as equally costly. In money, this is rarely so: a missed fraudster can cost much more than an extra check. Then you take a weighted version where recall matters more, or count the cost of errors directly. Before acceptance, agree on which data and which class the metric is calculated for, or the contractor will show an average over all classes, where a failure on the rare class is hidden. Such terms are easy to write into the pilot contract.
For a specialist
The F1 score is the harmonic mean of precision and recall: F1 = 2PR / (P + R) = 2TP / (2TP + FP + FN). True negatives (TN) are not in the formula, so the measure does not change when the number of easy negative examples grows. The general case Fβ sets the weight of recall through β and goes back to the effectiveness measure E from the classic book Information Retrieval (second edition, 1979), where the parameter α = 1/(β² + 1). In scikit-learn, multiclass variants average like this: micro over the total TP, FP and FN; macro as an unweighted mean over classes, sensitive to rare classes; weighted with weights by the number of examples per class, and then the result may fall outside the range between precision and recall; samples for the multilabel setting. If a class appears neither in the true labels nor in the predictions, the measure is undefined, and the library returns 0 with a warning by default. The value depends on the decision threshold, so the threshold is tuned on validation, and for a comparison without a threshold you look at the precision-recall curve.
Example
A bank’s security service compares two fraud detection models. The first has a higher share of correct answers, but its F1 on fraudulent payments is almost half as high: it simply calls almost everyone honest. They choose the second.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map