MMOOZZGG.com

Glossary

What is a confusion matrix?

In short

A confusion matrix is a table that compares a model’s answers with the correct answers, so you can see how many times and in which direction the model made a mistake.

Machine learning · confusion matrix

MMOOZZGG.com editorsChecked: 2 min read

For a child

You sort fruit: apples in one basket, pears in another. Then your mom checks and draws a little table. One cell holds the apples that went to the apples, another the pears that went to the pears. Two more cells are for the mix-ups: apples among the pears and pears among the apples. You see at once where you get confused more often. A computer is checked with the same kind of table.

In plain words

Take a spam filter. Each email falls into one of four cells. Spam the filter caught. Spam it let into the inbox. A good email it moved to spam by mistake. And a good email that stayed in place. A table of these four numbers shows not only the overall share of right answers but also the kind of mistakes: does the model more often miss what matters, or raise false alarms? With more than two possible answers, the table grows: it has as many rows and columns as there are classes, and you see which classes the model mixes up.

For business

A single quality number hides the main point, and this table shows it: which mistakes the model makes and how many. A manager can put a price on each cell. A missed claim costs a fine, and a false alarm costs an hour of staff time. Multiply the number of mistakes by their price, and you get the cost of the misses in money to compare with your current process. For a request classifier, the table may show, for example, that questions about refunds often go to the delivery team, so you see what to retrain. Ask your vendor to build it on your real data, not on demo data, and to review rare but costly classes separately. The same check applies to automated request handling before launch.

For a specialist

A confusion matrix is a square K×K table C for K classes, where the element C[i, j] is the number of items of true class i that the model put in class j. In scikit-learn, rows are true labels and columns are predicted ones; in the binary case C[0,0] is true negatives (TN), C[0,1] false positives (FP, type I error), C[1,0] false negatives (FN, type II error) and C[1,1] true positives (TP). Some papers and tools swap the axes, so always check the labels. Almost all threshold metrics come from the cells: accuracy, precision, recall, specificity, F1 score and the Matthews correlation coefficient (MCC). The normalize parameter divides the counts by the row sums (‘true’), so the diagonal shows each class’s recall, by the column sums (‘pred’), so it shows precision, or by the total (‘all’). The table is computed at a fixed threshold; changing the threshold gives a family of tables, from which the ROC curve is built. For multi-label tasks, you take a separate 2×2 table for each label.

Example

A model sorts support requests into five topics. The confusion matrix shows that it puts a quarter of payment questions under refunds. The team adds training examples where the two topics are easy to mix up, and the confusion drops clearly.

Related terms

Questions and answers

How do you read a confusion matrix? The diagonal holds the right answers, and every other cell is a mistake. A row shows where the model put the items of one true class.

A column shows which classes make up its answers with one label.

What are a false positive and a false negative? A false positive is an alarm without a reason: an honest payment marked as fraud. A false negative is a miss: a fraudulent payment passed as honest.

The two mistakes usually cost different amounts, and that cost decides how you tune the model.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us