Modular RAG: Treating the Pipeline as Swappable Parts
The modular framing, its published taxonomy, and what it buys you over a fixed pipeline.
Modular RAG is a framing that decomposes the RAG pipeline into independent modules and operators — retriever, reranker, generator, router, memory — so stages can be swapped and rewired without rebuilding a fixed retrieve-then-generate chain. Gao, Xiong, Wang and Wang (arXiv:2407.21059, July 2024) name the paradigm and the routing, scheduling and fusion mechanisms that replace a single linear pass. On this site it sits on the architectures ladder between Advanced RAG and Agentic RAG.
Is Modular RAG the same as Naive RAG or Advanced RAG?
Most confusion about Modular RAG comes from treating it as “Advanced RAG with more techniques.” The three labels name different control structures, not a scoreboard of features.
| Modular RAG is not… | Because |
|---|---|
| …Naive RAG | Naive RAG is a fixed retrieve-then-generate pass: one retrieval, one generation, no routing graph. Modular RAG makes the stages themselves reconfigurable. |
| …just Advanced RAG | Advanced RAG keeps a largely linear skeleton and bolts on pre-retrieval and post-retrieval optimisations (rewrite, expand, rerank, compress). Modular RAG changes whether that skeleton is fixed. |
| …a single product or database | It is an architectural framing. Any stack that exposes swappable retrieval, ranking and generation stages with explicit orchestration can be modular. |
| …the same as agentic RAG | Routing can be rules or a classifier. An agent that decides whether, what and how often to retrieve is a further control loop — see Agentic RAG. |
Advanced RAG improves a fixed pipe. Modular RAG asks whether the pipe must stay fixed when the query class changes.
How does Modular RAG work?
Modular RAG works by factoring the end-to-end job into modules — indexing, retrieval, generation, orchestration — with finer operators inside them (embed, expand the query, rerank, compress, fuse) that an orchestrator can assemble per query. Gao et al. (2024) organise the frame in three tiers: modules, sub-modules and operators; Sahin’s 2024 walkthrough of that paper is the clearest public restatement of the tiers.
The module set that keeps recurring across ranking pages is small:
- Retriever — selects candidate passages from a corpus (sparse, dense or hybrid).
- Reranker or filter — optional second pass that reorders or drops noise before generation.
- Generator — the LLM that writes from the refined context.
- Orchestration — routing, scheduling and fusion: which modules run, in what order, and how parallel results merge.
- Optional auxiliaries — memory of prior turns, summarisers, or planners for multi-step questions.
Orchestration is what makes the frame modular rather than a longer Advanced pipeline. The stage map those modules sit on is the RAG pipeline; this page defines the swappable framing, not every stage’s internals.
What flow patterns does Modular RAG use?
Modular RAG supports four published flow patterns beyond a single retrieve-then-generate pass — linear, conditional, branching and looping — named in Gao et al. (2024) and restated on the ranking explainers that track that paper. The pattern is the control graph; the modules are the nodes you can swap.
| Pattern | What changes | Failure that forces it | Module you typically swap or add | Open next |
|---|---|---|---|---|
| Linear | Fixed sequence of modules (expand → retrieve → rerank → generate) | A single path is enough; you only need better stages, not different graphs | Swap retriever, reranker or generator in place | Advanced RAG |
| Conditional | A router chooses which path runs | Query classes need different indexes or strategies (e.g. billing vs clinical) | Add a routing module / classifier | Architectures hub |
| Branching | Parallel retrieve or generate paths, then fuse | One query needs several expansions or sources at once | Add fusion after parallel retrievers | Multi-query / fusion |
| Looping | Iterate retrieve ↔ generate until a stop rule | One shot cannot compose an answer that needs several evidence steps | Add a critique / stop operator; may become agentic | Multi-hop · Agentic |
Adaptive or “active” retrieval — fetching again mid-generation when the model hits a knowledge gap — is a looping variant, not a fifth unrelated paradigm. If every query can share one linear path, you do not need the full modular control graph yet.
What are Modular RAG’s limitations?
Modular RAG’s costs are orchestration complexity, higher setup and operations burden, and loose coupling between stages that can leave retrieval and generation misaligned — the same trade-offs Meilisearch (2025), Sahin (2024) and Emergent Mind’s survey notes all surface.
- Setup and ops cost. More modules mean more infrastructure, interfaces and people who understand how they fit — Meilisearch’s guide states higher setup cost as the first disadvantage.
- Orchestration complexity. Root-causing a bad answer across router, scheduler and fusion is harder than debugging a single retrieve-then-generate pipe (Sahin’s challenges section; Meilisearch’s “complexity overload”).
- Loose coupling. A retriever and generator tuned separately can disagree: strong retrieval with a generator that ignores the context, or the reverse. Emergent Mind’s design-trade-offs note is the clearest public restatement — alignment is extra work, not free with the frame.
None of these are reasons to abandon Modular RAG; they are the bill you pay when the control graph itself must change.
When does Modular RAG beat a fixed pipeline?
Modular RAG beats a fixed Advanced pipeline when stages or paths must change without a full rebuild — swappability is the product, not a longer feature list.
- Swap one stage without redeploying the rest. Keep an embedding model you like and replace the retriever — the vignette Meilisearch uses for why modules exist.
- Route query classes to different indexes or strategies. Clinical vs billing questions, or keyword vs semantic paths for “running shoes under $100,” need a router, not one shared top-k.
- Upgrade stages in parallel. Separate teams can change retrieval and generation on different cadences when interfaces stay stable (Meilisearch’s “faster development” advantage).
If every query runs the same retrieve → rerank → generate path and never needs a different graph, Advanced RAG on a fixed pipe is enough. Modular earns its cost when the control graph itself must change — which is why it sits on the architectures map as a framing, not a single algorithm.
Where does Modular RAG fit among RAG architectures?
Modular RAG is the framing on the architectures ladder after Naive and Advanced and before Agentic: this page pins the modules and flow patterns; the stage-by-stage workflow lives on the RAG pipeline; runnable assembly lives on how to build a pipeline. Implementation code, evaluation playgrounds and agentic control-loop depth are out of scope here — they belong on those linked pages, not on a definitional node.
What is Modular RAG?
Modular RAG is a retrieval-augmented generation framing that breaks the pipeline into independent modules and operators — retriever, reranker, generator, router, memory — so stages can be swapped and rewired without rebuilding a fixed retrieve-then-generate chain. Gao et al. (arXiv:2407.21059, July 2024) name the paradigm and the routing, scheduling and fusion mechanisms that replace a single linear pass.
How is Modular RAG different from Advanced RAG?
Advanced RAG keeps a largely linear retrieve-then-generate skeleton and adds pre-retrieval and post-retrieval optimisations such as query rewriting, expansion, reranking and compression. Modular RAG changes whether that skeleton is fixed: stages become swappable modules under an orchestrator that can route, branch or loop. Advanced improves a fixed pipe; Modular asks whether the pipe must stay fixed.
What are the four Modular RAG flow patterns?
Gao et al. (2024) identify four common patterns: linear (fixed module sequence), conditional (a router chooses the path), branching (parallel retrieve or generate paths then fusion), and looping (iterate retrieve and generate until a stop rule). Adaptive or active retrieval mid-generation is a looping variant, not a separate paradigm.
When should I use Modular RAG instead of a fixed pipeline?
Use Modular RAG when you must swap a retriever or embedding model without redeploying generation, route different query classes to different indexes or strategies, or let teams upgrade stages on different cadences. If every query can share one retrieve → rerank → generate path, Advanced RAG on a fixed pipe is enough — Modular earns its orchestration cost when the control graph itself must change.
Does Modular RAG require an agent?
No. Routing can be rules or a classifier that picks among module paths. An agent that decides whether, what and how often to retrieve is a further control loop — Agentic RAG — not a requirement of the modular framing. See the Agentic RAG page when the decision itself must be model-driven.