Glossary
What is a bi-encoder
In short
A bi-encoder is a search design that turns a question and each document into vectors separately, then compares the vectors by closeness.
Search and RAG · bi-encoder
MMOOZZGG.com editorsChecked: 1 min read
For a child
Each pupil in a class has a card with favorite activities. The teacher filled in the cards in advance, back in September. When a new pupil comes, the pupil gets a card too, and you at once see who shares the same interests. You do not need to introduce the pupil to everyone one by one. A bi-encoder also makes cards for all documents in advance.
In plain words
Imagine a library with a huge number of articles. If you carefully compare a question with each one, you wait a long time for an answer. A bi-encoder solves this differently. All articles are turned into sets of numbers in advance, once, and stored in a database. When a question comes, only the question is turned into numbers, and the database is searched for the closest sets. This takes a fraction of a second. The price of speed: the question and the article never meet inside the model, so it sometimes misses fine differences in meaning. For example, “how to cancel an order” and “how to place an order” may end up close to each other.
For business
Almost any meaning-based search in corporate assistants rests on this scheme. The main costs are one-time: compute vectors for all documents and keep them in a vector database. After that each question is cheap, so the scheme handles large databases and many users. The pitfalls: when you change the model, the whole database is recomputed, and a model trained on general texts may poorly tell apart the terms of your industry. Accuracy is added in a second step, by reranking the results. The size of the database and the number of users decide whether one server is enough, and this is calculated when you choose hardware.
For a specialist
A bi-encoder (also called a dual encoder) encodes the query and the document independently, often with one network with shared weights (a Siamese architecture), and computes relevance with a simple function of the two vectors: the dot product or cosine. The advantage is that the corpus representations are computed in advance and indexed for approximate nearest neighbor search. Humeau et al. (2019) contrasted this scheme with the cross-encoder: the bi-encoder is faster but less accurate, because it does not see the interaction between query and document tokens. Dense Passage Retrieval (Karpukhin et al., 2020) showed that a dual encoder trained on a small number of question and passage pairs beats Lucene-BM25 by 9 to 19 points in top-20 accuracy. Such models are trained contrastively, with in-batch negatives and hard negatives. Middle-ground designs between the two schemes are the poly-encoder and late interaction in ColBERT.
Example
A support team keeps the vectors of all past requests. A new customer email is encoded in milliseconds, and the operator at once sees similar cases with ready solutions.
Related terms
Questions and answers
Are a bi-encoder and an embedding model the same thing?
Sources
Not sure whether your company needs this?
AI solutions map