Glossary
Who is Yoshua Bengio
In short
Yoshua Bengio is a deep learning researcher at the University of Montreal, the founder of the Mila institute and a winner of the 2018 Turing Award, together with Geoffrey Hinton and Yann LeCun.
People · Yoshua Bengio
MMOOZZGG.com editorsChecked: 2 min read
For a child
Imagine that every word has its own little house on a big city map. Words with a similar meaning live next door: “cat” next to “kitten”, “car” next to “bus”. Yoshua Bengio taught a computer to draw such a map by itself, by reading a great many books. Now programs understand that words can be close in meaning even when they are spelled in completely different ways.
In plain words
Bengio is a professor at the University of Montreal and the founder of Mila, the Quebec Artificial Intelligence Institute. In 2003 he and his colleagues proposed a neural network language model: each word turns into a set of numbers, and words with close meanings get close sets. The model learns to predict the next word and builds these representations along the way; today they are called embeddings. In 2014, with coauthors, he proposed a method in which a translation model itself looks at the needed words of the source phrase instead of squeezing the whole phrase into one vector. He shared the 2018 Turing Award with Hinton and LeCun.
For business
Two of Bengio’s ideas work in almost every project with language models. The first: words and documents are represented as numbers. So an AI assistant finds the right contract in a database even if you asked in other words than the text uses. Search quality depends on the texts the representation model was trained on, and on whether it knows your language and your industry terms. The second idea: the model itself chooses which parts of the text to look at. For a manager, this means the model reads a long document selectively and sometimes misses rare details. So test a document assistant on employees’ real questions before launch, not on examples from a sales deck.
For a specialist
The paper by Bengio, Ducharme, Vincent and Jauvin, “A Neural Probabilistic Language Model” (Journal of Machine Learning Research, vol. 3, 2003, pp. 1137–1155), fights the curse of dimensionality of n-gram models: each word gets a distributed representation, and the network learns these vectors and the probability of word sequences at the same time. A phrase that was not in the training data gets a high probability if it consists of words close to the words of known phrases. The paper by Bahdanau, Cho and Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate” (arXiv 1409.0473, 1 September 2014, presented at ICLR 2015), removes the bottleneck of the encoder-decoder architecture, where the source is squeezed into a fixed-length vector: for each word of the translation, the model makes a soft search for the relevant parts of the source. This alignment technique later grew into the attention mechanism of transformers. The 2018 ACM Turing Award was announced on 27 March 2019 and shared by Bengio, Hinton and LeCun.
Example
A law firm loaded thousands of contracts into an AI assistant. A lawyer asks where the company promised a fine for delay, while the text says “penalty for breach of the deadline”. The assistant still finds the clause, because the vectors of these phrases are close, as in Bengio’s 2003 model.
Related terms
Questions and answers
Sources
- Mila, Yoshua Bengiomila.quebec
- NYU Tandon School of Engineering, Professor Yann LeCun wins A.M. Turing Award, 27.03.2019engineering.nyu.edu
- Bengio, Ducharme, Vincent, Jauvin, A Neural Probabilistic Language Model, JMLR, 2003jmlr.org
- Bahdanau, Cho, Bengio, Neural Machine Translation by Jointly Learning to Align and Translate, arXiv 1409.0473, 2014arxiv.org
Where companies use it
Not sure whether your company needs this?
AI solutions map