MMOOZZGG.com

Glossary

What is modular RAG?

In short

Modular RAG is an approach in which a document search system is built from independent blocks that you rearrange for each task, like parts of a construction set.

Search and RAG · modular RAG

MMOOZZGG.com editorsChecked: 1 min read

For a child

You have a construction set. From the same parts you can build a car or a tower. If a wheel breaks, you change only the wheel, not the whole car. Modular RAG works in a similar way: a helper that looks for answers in books is built from separate parts, and you can replace or move any of them.

In plain words

In a simple setup, all steps go in one line: find, then answer. In a modular one, each step becomes a separate block: document search, product catalog search, question rewriting, answer checking, memory of the earlier conversation. A dispatcher block decides which of them a given question needs. A question about price goes to the catalog, a question about returns goes to the rules, and a greeting needs no search. You change blocks one at a time: you can put in another model or another search without rebuilding the whole system.

For business

A modular design pays off when an assistant works with several sources at once: the CRM, the 1C accounting system, a contract archive, the website. Each source connects as a separate block, and when you change the model or search vendor, you replace one part. This lowers your dependence on one vendor. Costs are higher than for a simple setup: you need routing, a call log and tests for each block. The main risk is complexity: without a log, it is hard to see which block gave a wrong answer. Before launch, agree who owns each block and how its quality is measured. Connections to accounting systems and the CRM are settled at the integration stage.

For a specialist

Modular RAG is described as the third paradigm in the survey by Gao et al. (2023) and developed in a 2024 paper by the same authors, which compares the pipeline to a LEGO set. The system breaks into independent modules and specialized operators, and routing, scheduling and fusion are added on top of the linear chain. The authors name four typical flows: linear, conditional, branching and looping. The survey lists the modules Search (external sources, databases and knowledge graphs), RAG-Fusion (several versions of the query), Memory, Routing (choice of source for a query), Predict (the model generates the context itself) and Task Adapter. Example patterns are Rewrite-Retrieve-Read and ITER-RETGEN, where retrieval and generation alternate in a loop. In this notation, naive and advanced RAG are a linear flow with a fixed set of modules.

Example

An online store builds an assistant from three blocks: catalog search, search in the delivery rules, and an order status request to the accounting system. The dispatcher sends each question to the right block.

Related terms

Questions and answers

How is modular RAG different from advanced RAG? Advanced RAG stays one line of steps, even with improvements. Modular RAG allows branches, loops and a choice of source for each question, and you can replace its blocks one by one.
Does a small company need modular RAG? If there is one source, such as a folder of company rules, a simple setup is enough. Modular RAG pays off when an assistant has to answer from several systems at once.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us