MMOOZZGG.com

Glossary

What are embeddings in a neural network?

In short

An embedding is a list of numbers by which a computer understands the meaning of a text or a picture and finds similar ones.

Language models · embedding

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine a big map where every word lives in its own little house. Similar words live close together: “cat” next to “kitten”, while “car” is far from them. The address of a house on this map is the embedding.

In plain words

A computer does not understand the meaning of words; it works with numbers. So a word, a phrase or a whole document is turned into a long list of numbers. The list is chosen so that texts with similar meaning get similar lists. Then the computer can find similar texts even if they use different words: “get my money back” and “request a refund” end up close together.

For business

Embeddings are needed when AI has to search the company’s documents: contracts, the knowledge base, correspondence. The texts are turned into embeddings once and stored in a vector database. For a customer’s question the system finds the fragments closest in meaning and gives them to the model. This is how RAG works. Search quality depends on the chosen embedding model and on how the documents are split, so this is checked in a pilot before adoption.

For a specialist

An embedding is a mapping of an object (a token, a sentence, a document, an image) to a dense vector of fixed size, trained so that geometric closeness reflects semantic closeness. Early methods trained word vectors from context (word2vec, GloVe). Modern models build contextual embeddings with a transformer and train them with contrastive losses on pairs of similar and different texts. Closeness is usually measured by cosine similarity or the dot product, and search over millions of vectors uses approximate methods such as HNSW.

Example

The query “how do I return an item” and the contract paragraph “procedure for processing a return” get close embeddings, although they share almost no words. So search finds the right paragraph.

Related terms

Questions and answers

How is an embedding different from a token? A token is a small piece of text, and an embedding is a numeric description of its meaning. A model first cuts text into tokens and then turns them into embeddings.
Where are embeddings used in business? In knowledge base search, in AI support assistants, in product recommendations and in finding similar customer requests.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us