Adaptive-RAG: Routing by Query Complexity

Adaptive-RAG (Jeong, Baek, Cho, Hwang and Park; NAACL 2024; arXiv:2403.14403) is a query-complexity router: a smaller language-model classifier picks no retrieval, single-step retrieval, or multi-step retrieval before the answer is generated. Official code is at github.com/starsuzi/Adaptive-RAG. This is not the same thing as binary adaptive retrieval (retrieve vs skip only), and it is not the loose vendor phrase “adaptive RAG” for any feedback-tuned pipeline.

How does Adaptive-RAG choose between no retrieval, single-step, and multi-step?

Adaptive-RAG treats strategy choice as a three-way decision matched to how hard the question is (Jeong et al., 2024, Figure 2 and §3.1–3.2):

1. No retrieval (label A). Answer with LLM(q) only — for queries the model’s parametric knowledge can already handle.
2. Single-step (label B). Retrieve once, then generate with LLM(q, d) — the ordinary retrieve-then-read path.
3. Multi-step (label C). Iterate retrieve-and-reason with context carried across steps (LLM(q, d, c)). In the paper’s experiments the multi-step branch follows the Trivedi et al. IRCoT-style iterative approach (Jeong et al., 2024, §4.2).

One-size-fits-all multi-step wastes calls on easy questions; one-size-fits-all single-step fails questions that need several documents. The router exists to pick the cheapest strategy that is still adequate. Depth on the C-branch loop itself lives at IRCoT; the single-step baseline cousin is Naive RAG.

How does the Adaptive-RAG query-complexity classifier work?

The classifier is a smaller LM that maps the query string to one of {A, B, C} before any retrieval strategy runs: o = Classifier(q) (Jeong et al., 2024, §3.2). The paper trains T5-Large with AdamW at learning rate 3e-5, picking the best validation epoch within 100 training iterations (§4.4).

Because no public A/B/C labels exist, the authors build silver training data automatically:

• Outcome-based labels. Run no-retrieval, single-step and multi-step solvers; if the simplest solver that gets the answer right succeeds, prefer that simpler label (ties break toward simpler strategies).
• Dataset inductive bias. Remaining unlabeled queries inherit B from single-hop datasets and C from multi-hop datasets.

Annotation used 400 sampled queries per dataset for bias labels plus predicted outcomes over 400 queries per dataset, with no overlap into the QA test set (§4.4). The published confusion matrix still shows real error modes: Multi misclassified as One about 31%, One as Multi about 23%, No as One about 47%, and No as Multi about 22% (Jeong et al., 2024, §5). Binary retrieve-or-skip gating without a multi-step tier is the sibling at adaptive retrieval.

What results did Adaptive-RAG publish?

Table 1 averages single-hop (SQuAD, Natural Questions, TriviaQA) and multi-hop (MuSiQue, HotpotQA, 2WikiMultiHopQA) sets with 500 test samples per dataset. Step and Time are relative to the single-step approach (= 1.00) (Jeong et al., 2024, Table 1, §4.3).

• FLAN-T5-XL (3B). Adaptive-RAG reaches EM 37.17, F1 46.94, Acc 42.10 at Step 2.17 / Time 3.60. Always multi-step scores EM 39.00, F1 48.85 at Step 4.69 / Time 8.81. Single-step sits at EM 34.83, F1 44.31.
• GPT-3.5 (Turbo instruct). Adaptive-RAG EM 37.97, F1 50.91, Acc 48.97 at Step 1.03 / Time 1.46 versus multi-step EM 38.13, F1 50.87 at Step 2.81 / Time 3.33.
• Oracle upper bound. With a perfect classifier, Adaptive-RAG on FLAN-T5-XL reaches EM 45.00, F1 56.28 at Step 1.28 / Time 2.11 — the gap to the learned classifier is the headroom the Limitations section flags.

Self-RAG appears in the same table with an explicit caveat that it uses a different base LLM (LLaMA2), so those rows are not an apples-to-apples bake-off — see Self-RAG and RAG benchmarks.

What does Adaptive-RAG cost compared with always multi-step?

Adaptive-RAG’s product claim is efficiency at near-multi-step accuracy: on FLAN-T5-XL it roughly halves relative Time versus always multi-step (3.60 vs 8.81) while staying within a few EM points (Jeong et al., 2024, Table 1). Table 3 shows why the mix matters in their experiment: predicted No (A) queries took 0.35 s (8.60% of samples), One (B) 3.08 s (53.33%), and Multi (C) 27.18 s (38.07%). Routing a simple query into C pays that multi-step wall time for nothing; routing a hard query into A fails silently. The classifier itself adds one smaller-LM forward pass before the chosen solver. Dollars per query on a private corpus are not published — measure Step, Time and EM together. Pipeline spend framing is at pipeline cost.

When should you use Adaptive-RAG, and when should you avoid it?

Adaptive-RAG earns its classifier when query traffic really mixes easy parametric questions, single-hop corpus lookups and multi-hop compositions. It is usually the wrong default when almost every query needs the same strategy.

• Use it on mixed-complexity traffic. When always-on IRCoT is too expensive and always-on single-step is too weak — the paper’s motivating distribution (Jeong et al., 2024, Introduction).
• Avoid it when complexity is uniform. If your logs are almost all multi-hop, implement multi-hop RAG (or IRCoT) directly; if they are almost all single-hop, stay on Naive RAG.
• Budget for classifier error. Silver labels can be wrong; only three discrete complexity levels are modeled; the oracle gap in Table 1 shows remaining headroom (Jeong et al., 2024, Limitations).

How does Adaptive-RAG differ from adaptive retrieval and Self-RAG?

Adaptive-RAG is a three-strategy router. Binary adaptive retrieval (Mallen et al. entity-frequency gates, Self-RAG Retrieve tokens, confidence gates) mainly decides whether to retrieve at all — covered on adaptive retrieval. Self-RAG additionally trains the generator to emit reflection tokens; Adaptive-RAG keeps a separate T5-Large classifier and swappable solvers, and compares to Self-RAG only with the different-base-LLM caveat in Table 1 (Jeong et al., 2024, §4.2). The multi-step branch is usually an IRCoT-style loop — mechanism depth at IRCoT, not on this page.

How do you implement Adaptive-RAG?

Implementation is a handoff checklist: train the complexity classifier, wire three solvers, route, and evaluate accuracy with Step/Time.

1. Build silver A/B/C labels. Prefer outcome-based labels with simpler-wins tie-break, then fill gaps with single-hop→B / multi-hop→C bias (Jeong et al., 2024, §3.2).
2. Train a small classifier. The paper’s instantiation is T5-Large; size ablations in the paper show smaller classifiers remain usable (§5).
3. Implement three solvers. No-retrieval LLM; single-step BM25+LLM (paper retriever is BM25); multi-step IRCoT-style branch.
4. Start from the official repo for fidelity. starsuzi/Adaptive-RAG; end-to-end wiring at building a RAG pipeline.