Glossary
What are embeddings in a neural network?
In short
An embedding is a list of numbers by which a computer understands the meaning of a text or a picture and finds similar ones.
Language models · embedding
MMOOZZGG.com editorsChecked: 1 min read
For a child
Imagine a big map where every word lives in its own little house. Similar words live close together: “cat” next to “kitten”, while “car” is far from them. The address of a house on this map is the embedding.
In plain words
A computer does not understand the meaning of words; it works with numbers. So a word, a phrase or a whole document is turned into a long list of numbers. The list is chosen so that texts with similar meaning get similar lists. Then the computer can find similar texts even if they use different words: “get my money back” and “request a refund” end up close together.
For business
Embeddings are needed when AI has to search the company’s documents: contracts, the knowledge base, correspondence. The texts are turned into embeddings once and stored in a vector database. For a customer’s question the system finds the fragments closest in meaning and gives them to the model. This is how RAG works. Search quality depends on the chosen embedding model and on how the documents are split, so this is checked in a pilot before adoption.
For a specialist
An embedding is a mapping of an object (a token, a sentence, a document, an image) to a dense vector of fixed size, trained so that geometric closeness reflects semantic closeness. Early methods trained word vectors from context (word2vec, GloVe). Modern models build contextual embeddings with a transformer and train them with contrastive losses on pairs of similar and different texts. Closeness is usually measured by cosine similarity or the dot product, and search over millions of vectors uses approximate methods such as HNSW.
Example
The query “how do I return an item” and the contract paragraph “procedure for processing a return” get close embeddings, although they share almost no words. So search finds the right paragraph.
Related terms
Questions and answers
Sources
Where companies use it
Not sure whether your company needs this?
AI solutions map