MMOOZZGG.com

Glossary

Who is Yoshua Bengio

In short

Yoshua Bengio is a deep learning researcher at the University of Montreal, the founder of the Mila institute and a winner of the 2018 Turing Award, together with Geoffrey Hinton and Yann LeCun.

People · Yoshua Bengio

MMOOZZGG.com editorsChecked: 2 min read

For a child

Imagine that every word has its own little house on a big city map. Words with a similar meaning live next door: “cat” next to “kitten”, “car” next to “bus”. Yoshua Bengio taught a computer to draw such a map by itself, by reading a great many books. Now programs understand that words can be close in meaning even when they are spelled in completely different ways.

In plain words

Bengio is a professor at the University of Montreal and the founder of Mila, the Quebec Artificial Intelligence Institute. In 2003 he and his colleagues proposed a neural network language model: each word turns into a set of numbers, and words with close meanings get close sets. The model learns to predict the next word and builds these representations along the way; today they are called embeddings. In 2014, with coauthors, he proposed a method in which a translation model itself looks at the needed words of the source phrase instead of squeezing the whole phrase into one vector. He shared the 2018 Turing Award with Hinton and LeCun.

For business

Two of Bengio’s ideas work in almost every project with language models. The first: words and documents are represented as numbers. So an AI assistant finds the right contract in a database even if you asked in other words than the text uses. Search quality depends on the texts the representation model was trained on, and on whether it knows your language and your industry terms. The second idea: the model itself chooses which parts of the text to look at. For a manager, this means the model reads a long document selectively and sometimes misses rare details. So test a document assistant on employees’ real questions before launch, not on examples from a sales deck.

For a specialist

The paper by Bengio, Ducharme, Vincent and Jauvin, “A Neural Probabilistic Language Model” (Journal of Machine Learning Research, vol. 3, 2003, pp. 1137–1155), fights the curse of dimensionality of n-gram models: each word gets a distributed representation, and the network learns these vectors and the probability of word sequences at the same time. A phrase that was not in the training data gets a high probability if it consists of words close to the words of known phrases. The paper by Bahdanau, Cho and Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate” (arXiv 1409.0473, 1 September 2014, presented at ICLR 2015), removes the bottleneck of the encoder-decoder architecture, where the source is squeezed into a fixed-length vector: for each word of the translation, the model makes a soft search for the relevant parts of the source. This alignment technique later grew into the attention mechanism of transformers. The 2018 ACM Turing Award was announced on 27 March 2019 and shared by Bengio, Hinton and LeCun.

Example

A law firm loaded thousands of contracts into an AI assistant. A lawyer asks where the company promised a fine for delay, while the text says “penalty for breach of the deadline”. The assistant still finds the clause, because the vectors of these phrases are close, as in Bengio’s 2003 model.

Related terms

Questions and answers

Why did Yoshua Bengio receive the Turing Award? Bengio shared the 2018 Turing Award with Geoffrey Hinton and Yann LeCun for their work in deep learning. The award was announced on 27 March 2019, and the three winners split the prize of USD 1 million.
What is Mila, the institute Bengio founded? Mila is the Quebec Artificial Intelligence Institute in Montreal. Bengio founded it and remains its scientific advisor, and he works as a professor at the University of Montreal.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us