MMOOZZGG.com

Glossary

What is a cross encoder

In short

A cross encoder is a model that reads a question and a document together, as one text, and returns one number: how well the document answers the question.

Search and RAG · cross-encoder

MMOOZZGG.com editorsChecked: 1 min read

For a child

A teacher checks whether an answer fits a question. She can look at the question and the answer separately. Or she can put them side by side and read them together, word by word. The second way takes longer, but there are fewer mistakes. A cross encoder always reads the question and the answer side by side, so it understands well whether they fit each other.

In plain words

There are two ways to compare a question with a text. You can turn each into numbers separately and compare the numbers: this is fast. Or you can glue the question and the text into one line and let the model read them together. Then it sees how each word of the question links to each word of the text and notices fine details: “can be returned” and “cannot be returned” are different things to it. This is more precise, but each pair needs its own run. So such a model does not search the whole database; it re-sorts a few dozen pieces that were already found.

For business

This model is the second step of search, used when an assistant finds almost the right thing but answers from the neighboring clause. It computes each “question and passage” pair, so it sorts not the whole database but a few dozen candidates, and the answer delay grows depending on the hardware. Open models run on your own server, while cloud models charge per request. Check whether the model was trained on Russian texts: English-language models handle industry vocabulary worse. Measure the effect with a simple comparison: the share of correct passages in the top five before and after. It is easy to build this measurement into the pilot plan.

For a specialist

A cross encoder encodes a pair of sequences jointly: the query and the document go into a transformer as one string with a separator, full self-attention works across both parts, and a classification head returns a relevance score. Humeau et al. (2019) compared both designs in their paper on poly-encoders: they note that a cross encoder is usually more accurate but too slow for practical use, and a poly-encoder sits in between. A classic example is BERT as a passage reranker by Nogueira and Cho (2019), which beat the previous result on MS MARCO by 27% in MRR@10 (relative gain). The cost is linear in the number of candidates, and document representations cannot be computed in advance, so the model sits at the second stage, after BM25 or a bi-encoder. A cross encoder also serves as a teacher: its scores label pairs for distillation into fast bi-encoders.

Example

A knowledge base search returned three dozen passages for a question about returning goods without a receipt. The cross encoder lifts the rule about exactly this case to the top, and general words about returns go down.

Related terms

Questions and answers

How is a cross encoder different from a bi-encoder? A bi-encoder turns the question and the document into vectors separately, so it is fast. A cross encoder reads them together and judges the link more precisely, but more slowly, so it is used only on a short list of candidates.
Are a cross encoder and a reranker the same thing? Most often a reranker is built as a cross encoder. But a large language model can also act as a reranker, and a cross encoder is also used for other tasks, for example to label pairs.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us