MMOOZZGG.com

Glossary

Who is Vladimir Vapnik?

In short

Vladimir Vapnik is a mathematician who, with Alexey Chervonenkis, created statistical learning theory and built the theory of the support vector machine (SVM).

People · Vladimir Vapnik

MMOOZZGG.com editorsChecked: 2 min read

For a child

Sometimes you learn the problems from the textbook by heart before a test, but the test gives new ones, and nothing works. Vladimir Vapnik found a way to tell in advance whether a machine really learned or just memorized the examples. He also found how to draw a line between two piles of toys so that there is as much free space as possible on each side.

In plain words

From the early 1960s, Vapnik worked at the Institute of Control Sciences with Alexey Chervonenkis. They looked for the answer to one question: when will a model trained on a limited number of examples work correctly on new ones? The answer was a theory in which a model’s complexity is measured by a special number, the VC dimension. If a model is too flexible for that number of examples, it memorizes them instead of the pattern. In 1990, Vapnik moved to Bell Labs in the US and built the theory of the support vector machine: it looks for the border between classes with the widest margin. This method is still part of the basic machine learning toolkit today.

For business

Vapnik’s work explains the main risk of any model in a company: it looks great on old data and makes mistakes on new data. The cause is overfitting, when the model is too complex for the number of examples. This leads to practical rules. Test the model on data it has not seen, such as the last quarter. Do not use a complex neural network where a simple model and a small table are enough. And a vendor’s promise of “almost no errors” means nothing without a description of the test sample. Back in the USSR, the methods of Vapnik and Chervonenkis were used for medical diagnosis together with the All-Union Cancer Center. How to catch such errors before launch is covered in the materials on AI risks.

For a specialist

At the Institute of Control Sciences, Vapnik and Chervonenkis developed the generalized portrait method for pattern recognition in 1962–1971. In 1968 they proved the necessary and sufficient conditions for the uniform convergence of frequencies to probabilities over an infinite class of events, a generalization of the law of large numbers. This gave the concept of VC dimension (Vapnik-Chervonenkis dimension), bounds on the convergence of empirical risk minimization methods, and the structural risk minimization method. The results were set out in the book “Theory of Pattern Recognition” (with Chervonenkis, Nauka, 1974). In 1990, Vapnik moved to AT&T Bell Laboratories and, based on the generalized portrait, built the theory of support vector machines (SVM): the classifier maximizes the margin between classes, and kernels (the kernel trick) move the task into a high-dimensional feature space. His main books in English are “Statistical Learning Theory” (Wiley, 1998) and “The Nature of Statistical Learning Theory” (Springer, 1999).

Example

A delivery service trained a model to predict courier delays, and on last year’s data it made almost no mistakes. An analyst followed Vapnik’s rule and tested it on the last month, which the model had not seen. The errors grew noticeably, and the model was simplified before launch.

Related terms

Questions and answers

What is the Vapnik-Chervonenkis (VC) dimension? VC dimension is a measure of how complex a class of models is: how many different ways the model can split examples. The higher it is for the same number of examples, the greater the risk that the model memorizes the data instead of the pattern.
Who invented the support vector machine? Vladimir Vapnik built the theory of the support vector machine on the generalized portrait method, which he created with Alexey Chervonenkis in 1962–1971. He developed SVM theory after 1990, when he worked at AT&T Bell Laboratories.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us