Glossary
What is modular RAG?
In short
Modular RAG is an approach in which a document search system is built from independent blocks that you rearrange for each task, like parts of a construction set.
Search and RAG · modular RAG
MMOOZZGG.com editorsChecked: 1 min read
For a child
You have a construction set. From the same parts you can build a car or a tower. If a wheel breaks, you change only the wheel, not the whole car. Modular RAG works in a similar way: a helper that looks for answers in books is built from separate parts, and you can replace or move any of them.
In plain words
In a simple setup, all steps go in one line: find, then answer. In a modular one, each step becomes a separate block: document search, product catalog search, question rewriting, answer checking, memory of the earlier conversation. A dispatcher block decides which of them a given question needs. A question about price goes to the catalog, a question about returns goes to the rules, and a greeting needs no search. You change blocks one at a time: you can put in another model or another search without rebuilding the whole system.
For business
A modular design pays off when an assistant works with several sources at once: the CRM, the 1C accounting system, a contract archive, the website. Each source connects as a separate block, and when you change the model or search vendor, you replace one part. This lowers your dependence on one vendor. Costs are higher than for a simple setup: you need routing, a call log and tests for each block. The main risk is complexity: without a log, it is hard to see which block gave a wrong answer. Before launch, agree who owns each block and how its quality is measured. Connections to accounting systems and the CRM are settled at the integration stage.
For a specialist
Modular RAG is described as the third paradigm in the survey by Gao et al. (2023) and developed in a 2024 paper by the same authors, which compares the pipeline to a LEGO set. The system breaks into independent modules and specialized operators, and routing, scheduling and fusion are added on top of the linear chain. The authors name four typical flows: linear, conditional, branching and looping. The survey lists the modules Search (external sources, databases and knowledge graphs), RAG-Fusion (several versions of the query), Memory, Routing (choice of source for a query), Predict (the model generates the context itself) and Task Adapter. Example patterns are Rewrite-Retrieve-Read and ITER-RETGEN, where retrieval and generation alternate in a loop. In this notation, naive and advanced RAG are a linear flow with a fixed set of modules.
Example
An online store builds an assistant from three blocks: catalog search, search in the delivery rules, and an order status request to the accounting system. The dispatcher sends each question to the right block.
Related terms
Questions and answers
Sources
Where companies use it
Not sure whether your company needs this?
AI solutions map