MMOOZZGG.com

Glossary

What is an embedding model?

In short

An embedding model is a neural network that turns a text, a picture or another object into an embedding, a set of numbers by which you can compare meaning.

Search and RAG · embedding model

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine a magic sticker fairy. She reads any note and puts a colored sticker on it. Notes about kittens get similar stickers, and notes about toy cars get very different ones. Then it is easy to put all the notes about kittens into one pile by their stickers. An embedding model is such a sticker fairy, only instead of colors it uses long rows of numbers.

In plain words

For a computer to search by meaning, each piece of text has to be turned into numbers. A separate neural network does this. It takes a phrase or a paragraph and returns a list of hundreds or thousands of numbers. It was trained in advance on a huge number of pairs of similar and different texts, so phrases close in meaning get close lists. The same model processes both documents and questions; otherwise they cannot be compared. For example, “how to extend a vacation” and “moving vacation days” end up close, although they share almost no words.

For business

This model decides whether the assistant finds the right document. Choose it by three questions: how well it understands your language and your industry, where it runs, and what re-indexing costs. A cloud model charges for the volume of text and sends documents outside; an open one can run on your own server, and the data stays in the company. Changing the model means re-indexing the whole base, so choose carefully. Public leaderboards are a useful guide, but the authors of the MTEB benchmark write themselves that no method wins in all tasks. Compare two or three models on your own documents and questions. When the base is very small, you may not need the model at all: CAG fits. Running such a model inside your own network is covered in the section on sovereign AI.

For a specialist

An embedding model is an encoder that maps an input to a dense vector of fixed size so that a similarity measure, usually cosine similarity, reflects semantic closeness. For texts a common scheme is Sentence-BERT (Reimers, Gurevych, 2019): a siamese or triplet network on top of a pretrained transformer with token pooling. The authors showed the scale of the gain: finding the most similar pair among 10,000 sentences takes about 65 hours with BERT, which must receive both sentences together, and about 5 seconds with Sentence-BERT. Modern models are trained with contrastive losses on pairs of a query and a relevant text with hard negatives, often with an instruction prefix for the task type. Quality is compared on MTEB (Muennighoff et al., 2022): 8 task types, 58 datasets, 112 languages; the authors found no method that dominates in all tasks. Practical selection parameters: vector size, maximum input length, languages, weight license and encoding speed.

Example

A company compares two models on real employee questions. The first misses on industry terms, the second finds the right document much more often, and the whole base is re-indexed with the second.

Related terms

Questions and answers

How is an embedding model different from a language model? A language model writes text. An embedding model does not write text; it turns text into a set of numbers for search and comparison.

A system that answers from documents usually uses both.

Can you run an embedding model on your own server? Yes. Many such models are open and small, so one graphics card is enough, sometimes even a CPU.

Then the text of your documents never leaves the company’s network.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us