MMOOZZGG.com

Glossary

Who was Alexey Ivakhnenko

In short

Alexey Ivakhnenko was a cybernetics scientist and the author of the Group Method of Data Handling (GMDH). From 1965, networks built on this method were possibly the first multilayer learning networks.

People · Alexey Ivakhnenko

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine a crafts contest in class. First everyone makes simple figures, and the teacher keeps the good ones. Then the class builds harder crafts from the good ones and again keeps those that worked. They repeat this many times until a beautiful ship comes out. Alexey Ivakhnenko taught a machine to build its hints the same way: step by step, keeping only what guesses the answer well.

In plain words

Ivakhnenko, a Soviet scientist, worked on control systems, cybernetics and machine learning. His main method, the Group Method of Data Handling, builds a model from data with no formula known in advance. First the machine tries many small formulas from pairs of input values, checks them on a separate part of the data and keeps the good ones. Then it builds the next layer from the good ones, and so on while quality grows. The result is a multilayer model that chose its own depth. The method is used in business analytics, medicine, ecology and sociology, wherever you need to find a link in measurements.

For business

Ivakhnenko’s method is valued because it gives a clear formula, not a black box. For a sales forecast or warehouse load, it picks the important factors by itself and drops the rest. This helps when data is small: hundreds of rows instead of millions, as at many mid-sized companies. The model is checked on a held-out part of the data, so the risk that it just memorizes the past is lower. It has a weak spot too: if a needed factor is missing from the data, the method will not invent it. So before a forecast goes live, the team finds out what sales, price and season data really exists in the records. Demand forecasting projects usually start with such a review.

For a specialist

The Group Method of Data Handling (GMDH) is described in papers by Ivakhnenko and Lapa (1965), Ivakhnenko (1968) and Ivakhnenko (1971, IEEE Transactions on Systems, Man and Cybernetics). The network grows inductively: nodes with polynomial functions (Kolmogorov-Gabor polynomials) are trained by regression, then extra nodes are pruned by quality on a separate validation set, and the task decides the number of layers and nodes. In the review “Deep Learning in Neural Networks: An Overview” (2014), Juergen Schmidhuber calls such networks possibly the first deep learning systems of the multilayer perceptron type and notes the 1971 paper with an 8-layer network. According to a KDnuggets note (2003), Ivakhnenko wrote 30 books and more than 400 papers, and the GMDH algorithm was built in 1968 on the ideas of a black box, natural selection and external addition. Close modern methods are neural architecture search and pruning.

Example

A beverage distributor wants to forecast demand by week, but its sales history is short. An analyst runs GMDH: the method tries combinations of price, weather and holidays, drops the extra ones on check weeks and gives a short formula the sales director can read.

Related terms

Questions and answers

What is Ivakhnenko’s GMDH? GMDH, the Group Method of Data Handling, builds a model from data layer by layer. It tries simple formulas, keeps those that work well on check data and builds the next layer from them.
Why is Ivakhnenko called a forerunner of deep learning? In a 2014 review, Juergen Schmidhuber called the GMDH networks described from 1965 possibly the first deep learning systems. As early as 1971, Ivakhnenko described a network of 8 layers.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us