MMOOZZGG.com

Glossary

What is a vector in machine learning?

In short

A vector in machine learning is an ordered list of numbers that a model uses to describe a word, a picture, a customer or another object, so it can calculate with the object and compare it with others.

Math of AI · vector

MMOOZZGG.com editorsChecked: 2 min read

For a child

Imagine you write down three numbers for every toy: how big it is, how soft it is and how bright it is. A teddy bear gets one set of three numbers, a plastic car gets another. From these numbers you can tell which toys are alike without even looking at them. Mathematicians call such a row of numbers a vector.

In plain words

A computer cannot work with meaning directly; it needs numbers. So any object is turned into a row of numbers, where each position has its own meaning. You can describe a flat like this: area, floor, year built, distance to the metro. If the rows of two flats are close, the flats are similar. Neural networks do the same with texts and pictures, only they use hundreds or thousands of numbers, and a person can no longer say what each one means. But by how close these rows are, the model finds similar documents and products.

For business

For business, vectors are the basis of search by meaning and of recommendations. Company documents, product cards and customer requests are turned into number descriptions and kept in a vector database. When an employee asks a question, the system finds the passages closest in meaning, even if they use other words. The length of each description and the number of documents set the memory you need and the cost of storage. There is a risk too: a study by Morris et al. (EMNLP 2023) showed that much of the original text can be recovered from such a description, and in a test on medical records even full names came back. So protect these databases the same way as the documents themselves. Test how this works on your data in a pilot of an AI assistant.

For a specialist

A vector in machine learning is an ordered array of numbers in which each element is defined by its index; formally, an element of an n-dimensional real space. The textbook Deep Learning (Goodfellow, Bengio, Courville, 2016) places it in the series scalar, vector, matrix, tensor. Linear algebra operations are defined on such arrays: addition, multiplication by a number, dot product and norm. Almost every calculation in a neural network is expressed through them: a layer’s input, a neuron’s weights, token embeddings, the gradient of the loss function. The closeness of two objects is measured by cosine similarity or Euclidean distance. Mikolov, Yih and Zweig (NAACL 2013) showed that word representations encode relations as constant shifts (vector offset): “King” minus “Man” plus “Woman” lands very close to the representation of “Queen”. In modern models, embeddings have hundreds or thousands of components, and the search for nearest neighbors among millions of records uses approximate methods.

Do not confuse. Do not confuse it with a vector in the sense of “direction of development” or with vector graphics: here it means the list of numbers a model works with.

Example

A store turns its product descriptions into number descriptions. A shopper searches for “warm jacket for the city”, and the system shows parkas and down coats that have no word “jacket” in their names but whose vectors are close to the query.

Related terms

Questions and answers

What is a vector in a neural network? It is a list of numbers the network uses to describe an object: a word, a picture, a sound. The network does all its calculations with such lists, and similar objects get close numbers.
How is a vector different from an embedding? The first is a math concept: any ordered set of numbers. An embedding is a special case of it, trained so that close numbers reflect close meaning.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us