MMOOZZGG.com

Glossary

What are precision and recall?

In short

Precision and recall are two quality metrics of a model. Precision shows what share of the found items is correct. Recall shows what share of the needed items the model found.

Machine learning · precision and recall

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine you need to take all the red blocks out of a big box. If your hand holds only red ones, you were careful. If not a single red one is left in the box, you were thorough. Sometimes you are careful but miss half, or you grab all the red ones together with a couple of orange ones. A computer is checked the same way: how much extra it took and how much it missed.

In plain words

Take a mail filter. Of 100 emails it sent to spam, 90 really are spam. That is the first metric, precision. Of all the spam emails in a day, it caught, say, eight out of ten. That is the second, recall. One pulls the other in the opposite direction. If the filter grabs everything suspicious, recall grows, but needed emails end up in spam. If it becomes careful, important emails stay in place, but more junk reaches you. The setting is chosen by the cost of each error.

For business

One overall quality number in a contractor’s presentation says almost nothing. A manager needs to know which error costs more. A missed fraudulent payment or a complaint that threatens a lawsuit costs money, so recall matters more there, even at the price of extra checks. A false alarm, when an honest customer is blocked or a manager spends an hour on an empty request, hurts revenue and reputation, and there precision matters more. Both metrics are measured on your real data and separately for each important class, not on demo examples. Before the contract, write down target values for both metrics and the threshold at which the AI hands the decision to a person. Agree these thresholds before the pilot starts.

For a specialist

For binary classification, from the confusion matrix: precision = TP / (TP + FP), recall (sensitivity) = TP / (TP + FN), where TP, FP and FN are true positive, false positive and false negative answers. Both values depend on the decision threshold. Sweeping the threshold gives the precision-recall curve, and average precision summarizes it. The weighted harmonic mean of the two metrics is called the F-measure: in Fβ, the parameter β sets the relative weight of recall, and β = 1 gives F1. For multiclass tasks, the metrics are averaged across classes (macro, micro, weighted). With a rare positive class, the share of correct answers (accuracy) is misleading, so people look at this pair instead. In Russian, accuracy is sometimes even called by the same word as precision, which adds to the confusion. In search and RAG, the variants precision@k and recall@k are used on the first k returned documents. The formulas and implementations are in the scikit-learn guide.

Example

A bank tunes a model that looks for fraudulent transfers. With high recall, it catches almost all fraudsters but sometimes holds honest payments, so an employee checks disputed cases right away.

Related terms

Questions and answers

Which matters more: precision or recall? It depends on the cost of an error. If a miss costs more than a false alarm, as in fraud detection, recall matters more.

If a false alarm costs more, as when customers get blocked, precision matters more.

How is precision different from accuracy? Accuracy is the share of all the model’s answers that are correct. Precision is counted only among the items the model called positive, so with a rare class these numbers can differ a lot.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us