Glossary
Who is Ashish Vaswani
In short
Ashish Vaswani is a researcher and the first author of the 2017 paper “Attention Is All You Need”, which describes the transformer.
People · Ashish Vaswani
MMOOZZGG.com editorsChecked: 1 min read
For a child
When you read a fairy tale, you know who “he” is at the end of a sentence, because in your mind you look back at the right words. Ashish Vaswani and other researchers taught a computer to do the same: to notice which words in a text are linked to each other. This idea became the base of programs that translate, write and answer questions today.
In plain words
Vaswani worked in the Google Brain team. In 2017, with seven coauthors, he published the paper “Attention Is All You Need”. It describes the transformer: a neural network that processes all the words of a sentence at once and decides, for each word, which other words to pay attention to. Earlier models read text one word at a time, which was slow. The transformer learns faster and keeps the meaning of long phrases more accurately. When the translator on your phone matches the words of a long sentence correctly, it relies on this idea.
For business
Almost all language models that companies connect today, from support chatbots to document search, are built on the transformer from the paper by Vaswani and his coauthors. For a manager, this has practical results. The cost of running a model depends on the length of the text in tokens: the attention mechanism compares words in pairs, so long documents cost more and take longer. The design is public, so many developers release similar models, and a company can choose between a cloud service and an open model on its own server. The risk is that a model makes confident mistakes if it does not get the data it needs. A separate page explains how the layers of AI in a company work, from data to actions.
For a specialist
The paper “Attention Is All You Need” (Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, Polosukhin) was posted to arXiv on 12 June 2017 and appeared in the NIPS 2017 proceedings (Advances in Neural Information Processing Systems 30). Vaswani is listed with a Google Brain affiliation. In a footnote, the authors write that the contribution was equal and the order of names random; Ashish, together with Illia Polosukhin, is credited with designing and building the first transformer models. The design drops recurrent and convolutional layers and relies only on the attention mechanism: an encoder and a decoder made of self-attention layers and fully connected networks. On the WMT 2014 machine translation task, the model scored 28.4 BLEU for English to German and 41.8 BLEU for English to French; training the second took 3.5 days on eight GPUs. BERT, GPT and today’s large language models (LLM) grew out of this design.
Example
A lawyer uploads a contract of several dozen pages to an assistant and asks it to find the termination terms. A transformer model matches a clause at the end of the document with the definitions at the start, so it answers with a reference to the right section.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map