Glossary
What is a vector in machine learning?
In short
A vector in machine learning is an ordered list of numbers that a model uses to describe a word, a picture, a customer or another object, so it can calculate with the object and compare it with others.
Math of AI · vector
MMOOZZGG.com editorsChecked: 2 min read
For a child
Imagine you write down three numbers for every toy: how big it is, how soft it is and how bright it is. A teddy bear gets one set of three numbers, a plastic car gets another. From these numbers you can tell which toys are alike without even looking at them. Mathematicians call such a row of numbers a vector.
In plain words
A computer cannot work with meaning directly; it needs numbers. So any object is turned into a row of numbers, where each position has its own meaning. You can describe a flat like this: area, floor, year built, distance to the metro. If the rows of two flats are close, the flats are similar. Neural networks do the same with texts and pictures, only they use hundreds or thousands of numbers, and a person can no longer say what each one means. But by how close these rows are, the model finds similar documents and products.
For business
For business, vectors are the basis of search by meaning and of recommendations. Company documents, product cards and customer requests are turned into number descriptions and kept in a vector database. When an employee asks a question, the system finds the passages closest in meaning, even if they use other words. The length of each description and the number of documents set the memory you need and the cost of storage. There is a risk too: a study by Morris et al. (EMNLP 2023) showed that much of the original text can be recovered from such a description, and in a test on medical records even full names came back. So protect these databases the same way as the documents themselves. Test how this works on your data in a pilot of an AI assistant.
For a specialist
A vector in machine learning is an ordered array of numbers in which each element is defined by its index; formally, an element of an n-dimensional real space. The textbook Deep Learning (Goodfellow, Bengio, Courville, 2016) places it in the series scalar, vector, matrix, tensor. Linear algebra operations are defined on such arrays: addition, multiplication by a number, dot product and norm. Almost every calculation in a neural network is expressed through them: a layer’s input, a neuron’s weights, token embeddings, the gradient of the loss function. The closeness of two objects is measured by cosine similarity or Euclidean distance. Mikolov, Yih and Zweig (NAACL 2013) showed that word representations encode relations as constant shifts (vector offset): “King” minus “Man” plus “Woman” lands very close to the representation of “Queen”. In modern models, embeddings have hundreds or thousands of components, and the search for nearest neighbors among millions of records uses approximate methods.
Do not confuse. Do not confuse it with a vector in the sense of “direction of development” or with vector graphics: here it means the list of numbers a model works with.
Example
A store turns its product descriptions into number descriptions. A shopper searches for “warm jacket for the city”, and the system shows parkas and down coats that have no word “jacket” in their names but whose vectors are close to the query.
Related terms
Questions and answers
Sources
- Goodfellow, Bengio, Courville, Deep Learning, chapter 2 Linear Algebra, MIT Press, 2016deeplearningbook.org
- Mikolov, Yih, Zweig, Linguistic Regularities in Continuous Space Word Representations, NAACL 2013aclanthology.org
- Morris et al., Text Embeddings Reveal (Almost) As Much As Text, EMNLP 2023arxiv.org
Where companies use it
Not sure whether your company needs this?
AI solutions map