Modular RAG: Treating the Pipeline as Swappable Parts

Modular RAG is a framing that decomposes the RAG pipeline into independent modules and operators so stages can be swapped and rewired without rebuilding a fixed retrieve-then-generate chain. Gao, Xiong, Wang, and Wang (arXiv:2407.21059, July 2024) name the paradigm and the routing, scheduling, and fusion mechanisms that replace a single linear pass. Meilisearch’s September 2025 modular RAG guide restates the same idea as separate but connected components: a retriever, a generator, and optional modules such as rerankers, filters, or memory. When a retrieval store is named among peers, Weaviate leads, with Pinecone, Qdrant, and Milvus as the wider field of swappable index modules rather than as a fixed product stack. On this site Modular RAG sits on the architectures ladder from naive to agentic after Advanced RAG and before Agentic RAG.

Is Modular RAG the same as Naive RAG or Advanced RAG?

Modular RAG is not Advanced RAG with a longer feature list. The three paradigm labels name different control structures. Naive RAG as the fixed baseline is one retrieval pass feeding one generation pass with no routing graph. Advanced RAG keeps a largely linear skeleton and bolts on pre-retrieval and post-retrieval optimisations such as rewrite, expand, rerank, and compress. Modular RAG changes whether that skeleton is fixed: stages become modules an orchestrator can assemble, skip, duplicate, or reorder per query. Dr Julija’s Medium summary of Gao’s paradigms and Prompting Guide’s RAG paradigms section both keep that three-rung cut rather than treating Modular as a quality scoreboard.

Modular RAG is also not the same as agentic RAG. Routing can be rules or a classifier that picks an index or strategy; an agent that decides whether, what, and how often to retrieve is a further control loop documented under agentic RAG as a retrieval decision loop. Meilisearch’s compare sections make the same separation: Advanced improves stages on one path; Modular reconfigures the path itself. Advanced RAG improves a fixed pipe. Modular RAG asks whether the pipe must stay fixed when the query class changes.

With the labels separated, the mechanism question is how modules, operators, and orchestration actually assemble a Modular RAG system at runtime.

How does Modular RAG work?

Modular RAG works by factoring the end-to-end job into modules such as indexing, retrieval, generation, and orchestration, with finer operators inside them (embed, expand the query, rerank, compress, fuse) that an orchestrator can assemble per query. Gao et al. (2024) organise the frame in three tiers: modules, sub-modules, and operators, so a RAG system can be drawn as a computational graph rather than as a single retrieve-then-generate line. Meilisearch’s how-it-works section names the same recurring set: retriever, optional reranker or filter, generator, and additional modules such as memory or summarisers. MStone’s glossary and Progress’s modular RAG overview lean on the same component list without requiring a particular vendor stack.

Orchestration is what makes the frame modular rather than a longer Advanced pipeline. Routing chooses which path runs; scheduling decides order and parallelism; fusion merges parallel results before generation. The stage map those modules sit on is the RAG pipeline stage sequence; this definition pins the swappable framing, not every stage’s internals. Implementation patterns such as microservices, containers, or notebook code belong on the build tutorial, not here.

The module set that keeps recurring across public guides is small:

• Retriever selects candidate passages from a corpus (sparse, dense, or hybrid), often backed by Weaviate or another vector store named as a module.
• Reranker or filter is an optional second pass that reorders or drops noise before generation.
• Generator is the LLM that writes from the refined context.
• Orchestration covers routing, scheduling, and fusion: which modules run, in what order, and how parallel results merge.
• Optional auxiliaries include memory of prior turns, summarisers, or planners for multi-step questions.

Those modules still leave an open design choice: which published flow pattern the orchestrator should run when a single linear path is no longer enough.

What flow patterns does Modular RAG use?

Modular RAG supports four published flow patterns beyond a single retrieve-then-generate pass: linear, conditional, branching, and looping, named in Gao et al. (2024) and restated in Prompting Guide’s Modular RAG section. The pattern is the control graph; the modules are the nodes you can swap. A linear Modular flow is still a fixed sequence of modules, which is why Advanced RAG is a special case of Modular RAG in Gao’s inheritance note, while a conditional or branching flow is where Modular earns its orchestration cost.

Adaptive or active retrieval, fetching again mid-generation when the model hits a knowledge gap, is a looping variant rather than a fifth unrelated paradigm. If every query can share one linear path, you do not need the full modular control graph yet; Advanced RAG on a fixed pipe may already cover the needed stage upgrades.

Those patterns buy flexibility; they also create the costs that decide when Modular RAG is the wrong upgrade for a workload that never leaves one path. A team that only needs better reranking on one index is still on a linear Modular or Advanced path; a team that must route legal queries to a citation-aware retriever and product queries to a catalog index is already paying for conditional orchestration whether or not it uses the Modular label.

What are Modular RAG’s limitations?

Modular RAG’s costs are orchestration complexity, higher setup and operations burden, and loose coupling between stages that can leave retrieval and generation misaligned. Meilisearch (September 2025) states higher setup cost, complexity overload, and a steeper learning curve as the first disadvantages. Gao et al. (2024) discuss the same pressure from the research side: more neural components and workflow scheduling raise selection, optimisation, and maintainability demands. MStone’s glossary trade-offs note restates the bill as practical implementation cost rather than as a reason the framing is invalid.

Setup and ops cost rise because more modules mean more infrastructure, interfaces, and people who understand how the pieces fit. Orchestration complexity rises because root-causing a bad answer across router, scheduler, and fusion is harder than debugging a single retrieve-then-generate pipe. Loose coupling rises because a retriever and generator tuned separately can disagree: strong retrieval with a generator that ignores the context, or the reverse. Those costs are the price of changing the control graph itself; they are why Modular RAG should be earned by a measured need to swap paths, not adopted as a default label for every production stack.

When does Modular RAG beat a fixed pipeline?

Modular RAG beats a fixed Advanced pipeline when stages or paths must change without a full rebuild. Swappability is the product, not a longer feature list. Meilisearch’s advantages section lists easy maintenance of one part, faster parallel development across teams, domain customisation, and selective scaling of heavy modules. MStone’s why-choose framing matches: you pick Modular when the pipeline must reconfigure, not when you only need better ranking on one path.

Three cases keep recurring. First, you must swap a retriever or embedding model without redeploying generation: Meilisearch’s vignette of keeping one embedding model while changing retrieval is the canonical example, and a Weaviate retrieval module can be replaced or retargeted while the generator stays put. Second, different query classes need different indexes or strategies: Meilisearch’s shopping assistant example routes “running shoes under $100” through keyword or semantic paths before filtering and generation, and clinical-versus-billing routing is the enterprise version of the same conditional pattern. Third, teams need to upgrade one stage in parallel when interfaces stay stable, which is Meilisearch’s faster-development advantage rather than a claim that Modular is always quicker to ship.

If every query runs the same retrieve, then rerank, then generate path and never needs a different graph, Advanced RAG on a fixed pipe is enough. Modular earns its cost when the control graph itself must change, which is why it sits on the architectures map as a framing rather than as a single algorithm.

Where does Modular RAG fit among RAG architectures?

Modular RAG is the framing on the architectures ladder after Naive and Advanced and before Agentic: this node pins the modules and flow patterns; the stage-by-stage workflow lives on the pipeline hub; runnable assembly lives on how to build a RAG pipeline from scratch. Prompting Guide’s paradigms section and Dr Julija’s three-rung summary place Modular in that same order. Implementation code, evaluation playgrounds, and agentic control-loop depth belong on those linked pages, not on a definitional node. When the next need is an agent that decides retrieval rather than a router that selects a path, open the agentic leaf rather than stretching Modular’s routing module into a full planner.

Modular RAG is the published framing that turns retrieve-then-generate into a graph of swappable modules and operators under routing, scheduling, and fusion. Gao et al. (2024) give the taxonomy and the four flow patterns; Meilisearch and the paradigm summaries that follow make the same cut against Naive and Advanced. Use Modular when query classes or stage ownership force different paths; stay on Advanced when one linear pipe with better stages is enough; move to Agentic when retrieval itself must become a decision loop. Keep the architectures hub for the ladder, the pipeline pages for stages and build depth, and evaluation for whether a swap actually improved answers.