Corrective RAG (CRAG): Grading Retrieval Before Generating

Corrective RAG (CRAG) is the Yan et al. (2024) architecture that grades retrieved documents with a lightweight evaluator and triggers Correct refine, Incorrect web fallback, or Ambiguous blend before generation. It trades an extra evaluate-and-correct step, and sometimes a web call, for robustness when retrieval is wrong. Shi-Qi Yan and Jia-Chen Gu share equal first authorship with Yun Zhu and Zhen-Hua Ling on Corrective Retrieval Augmented Generation (arXiv:2401.15884). The pattern is plug-and-play with stock generators, unlike reflection-token training. The same four letters also name Meta’s Comprehensive RAG Benchmark, a different job covered under the name collision below. Orientation across sibling patterns is at RAG architectures; the primary-source card sits in RAG research papers.

What is Corrective RAG (CRAG)?

Corrective Retrieval Augmented Generation (CRAG) inserts a retrieval-quality gate between retrieve and generate so the generator is not fed irrelevant or misleading top-k hits the way naive retrieve-then-generate does (Yan et al., 2024, abstract). Official code for the paper stack is published at github.com/HuskyInSalt/CRAG. Meilisearch’s Corrective RAG guide and GeeksforGeeks’ CRAG explainer open on the same definition: an evaluator scores the retrieval set, then one of three corrective actions runs before the answer is written.

The failure CRAG targets is specific. When the retriever is wrong, prepending those documents can worsen hallucinations rather than fix them (Yan et al., 2024, Introduction and Figure 1). Practitioners meet that surface as a wrong chunk retrieved or as hallucination despite context. CRAG’s answer is to grade and correct the retrieval set before generation, not to retrain the generator. The indiscriminate top-k baseline it hardens is naive RAG.

CRAG is not Meta’s Comprehensive RAG Benchmark. That evaluation suite also uses the acronym CRAG (Yang et al., Meta, arXiv:2406.04744; KDD Cup 2024) and measures open-domain factual RAG with mock search APIs. Same letters, different papers, different jobs. Scored depth for the Meta suite lives at RAG benchmarks.

How does Corrective RAG grade retrieval before generating?

CRAG scores every retrieved document against the query with a fine-tuned T5-large retrieval evaluator (about 0.77B parameters), producing a relevance score on [-1, 1] per query-document pair (Yan et al., 2024, §4.2). Those scores are compared to an upper and a lower threshold. The outcome triggers exactly one of three actions before generation runs.

The three actions are defined in Yan et al. (2024, §4.3) and restated across Meilisearch, GeeksforGeeks, and DataCamp’s LangGraph tutorial. Correct fires when at least one document scores above the upper threshold: CRAG keeps the internal corpus hits and runs decompose-then-recompose knowledge refinement, splitting documents into strips, re-scoring strips with the same evaluator, dropping weak strips (paper filter threshold -0.5, top strips kept), and concatenating what remains (Yan et al., 2024, §4.3-4.4; Appendix B.3). Incorrect fires when all documents score below the lower threshold: CRAG discards the corpus hits and replaces them with web search knowledge, rewriting the query into keywords (the paper used a ChatGPT rewrite prompt) and refining returned pages the same way (Yan et al., 2024, §4.3, §4.5; Google Search API in the paper’s experiments). Ambiguous fires when scores fall between the thresholds: CRAG blends refined internal knowledge with web knowledge so neither source carries the answer alone.

Thresholds are set empirically per dataset in Appendix B.3. PopQA used (0.59, -0.99), PubHealth and Arc-Challenge used (0.5, -0.91), and Biography used (0.95, -0.91). On PopQA, the T5 evaluator’s accuracy at judging overall retrieval quality was 84.3%, ahead of the ChatGPT prompt baselines the authors compared (Yan et al., 2024, §5.5). How information-retrieval metrics quantify the same failure surface is at measuring retrieval. End-to-end LangGraph walkthroughs belong at building a RAG pipeline.

What are Corrective RAG’s capabilities and limits?

No CRAG capability ships without the ceiling that rides with it. Each row below pairs what the pattern gives you with the limit that makes the published accuracy gain usable, so a production team can see what a paper delta does not buy on a private corpus.

Meilisearch’s CRAG guide and DataCamp’s tutorial both warn that evaluator maintenance, web quality, and extra steps raise cost. Yan et al. (2024, §6) state the evaluator must be fine-tuned. The matrix below consolidates those ceilings beside the capability each one gates.

The two rows that surprise teams first are the mandatory evaluator fine-tune and the web path’s hidden latency. Both get their own treatment in the cost section rather than a single cell. Candidate stores that feed the evaluator, including Weaviate, Qdrant, Pinecone, and Chroma, still need the same grade-then-act loop; the store choice does not remove the evaluator ceiling.

Published SelfRAG-LLaMA2-7b gains (CRAG versus RAG: +7.0 percentage points on PopQA, +14.9 FactScore on Biography, +36.6 percentage points on PubHealth, +15.4 percentage points on Arc-Challenge) are measured under the paper’s Contriever retrieval setup (Yan et al., 2024). They are evidence that the gate helps when retrieval quality varies, not a guarantee that the same deltas appear on a private corpus with a different retriever or threshold schedule. Retune Appendix B.3-style thresholds on your labelled set before treating any row as a production SLA.

What does Corrective RAG cost in latency and compute?

Yan et al. publish a generation-phase overhead estimate on PopQA (Table 6). Against standard RAG, CRAG rises from 26.5 to 27.2 TFLOPs per token and from 0.363 s to 0.512 s average executing time per instance. Against Self-RAG, Self-CRAG is reported at 27.2-80.2 TFLOPs per token and 0.908 s versus Self-RAG’s 26.5-132.4 TFLOPs and 0.741 s (Yan et al., 2024, Table 6).

Read that table with the authors’ own caveat: retrieval and data-processing stages are not included, so web search, page fetch, and strip filtering sit outside those numbers. When the Incorrect or Ambiguous action fires a web call, production latency is higher than Table 6 and is not published as a single end-to-end figure in the paper. That is the honest cost of the escape hatch: modest generation overhead on the Correct path, and an unbound web path when retrieval fails. Where multi-step RAG budgets break in practice is covered at RAG latency failures.

Meilisearch and DataCamp describe the same structural cost in qualitative terms: more steps mean more infra cost even when the published TFLOPs bump looks small. The about 2.6% TFLOPs rise from RAG to CRAG on the published PopQA estimate (26.5 to 27.2) is not a licence to ignore web latency when Incorrect fires.

Production teams that only budget the Table 6 generation delta underprice Incorrect and Ambiguous traffic. A low Incorrect rate still dominates p95 when each web call fetches and strips pages outside the published timing. Log action mix (Correct versus Incorrect versus Ambiguous) beside latency if you ship CRAG, so you can see whether the escape hatch is rare or the main path.

How does Corrective RAG compare to Self-RAG?

Both patterns add a quality-control loop; they put the critic in different places. CRAG grades retrieved documents with an external T5 evaluator and can replace a bad retrieval set with web knowledge without retraining the generator (Yan et al., 2024). Self-RAG (Asai et al., 2023; ICLR 2024) trains the generator with reflection tokens to decide when to retrieve and to critique passages and outputs, so instruction-tuning is part of the product. Yan et al. report that Self-RAG’s advantage collapses on untuned LLaMA2-hf-7b while CRAG stays competitive, because CRAG does not require the generator to emit special critic tokens (§5.3).

The paper also couples the two as Self-CRAG. On SelfRAG-LLaMA2-7b, Self-CRAG versus Self-RAG improves PopQA by 6.9 percentage points, Biography FactScore by 5.0, and PubHealth by 2.4 percentage points (Yan et al., 2024, §5.3). Depth on reflection tokens is at Self-RAG.

Agentic RAG is a broader control loop that may choose whether, what, and how many times to retrieve, and which tools to call. CRAG is one corrective gate; agentic systems can embed that gate among other decisions on agentic RAG. Speculative RAG optimises draft-and-verify speed rather than a retrieval grader for trust, a different target covered on speculative RAG. Which pattern wins for a given corpus is not settled here: start from the failure you actually have.

When should you use Corrective RAG?

Use Corrective RAG when your dominant failure is bad or incomplete retrieval that still reaches the generator, and you can afford an evaluate-and-correct step plus an occasional web call. That fit matches open-domain or stale static corpora, high cost of a confidently wrong answer, and a retriever you want to keep while adding a plug-in gate (the shape Meilisearch, GeeksforGeeks, and DataCamp describe as when CRAG earns its complexity).

Skip or constrain it when latency budgets cannot absorb even the published generation bump (+0.149 s on the paper’s PopQA CRAG versus RAG timing) plus an unbound web path; when you cannot fine-tune or monitor an external evaluator (Yan et al., 2024, §6); or when private documents must not leave the perimeter via web search. Implementing the graph end-to-end is how to build the pipeline.

CRAG does not always search the web. Only Incorrect and Ambiguous paths call web search; Correct keeps refined internal knowledge. That distinction matters for privacy reviews and for cost models that treat every query as a web call.

Is Corrective RAG the same as the CRAG benchmark?

No. Corrective RAG / CRAG (Yan, Gu, Zhu and Ling, 2024; arXiv:2401.15884) is an architecture that grades and corrects retrieval before generation. The Comprehensive RAG Benchmark, also abbreviated CRAG (Yang et al., Meta; arXiv:2406.04744; KDD Cup 2024), is an evaluation suite for open-domain factual RAG with mock search APIs.

Same four letters; different papers; different jobs. Live search for “yan et al crag” surfaces both, which is why the collision belongs on the architecture profile rather than only on the benchmark leaf. What the Meta suite measures is at what RAG benchmarks measure; the architecture paper card remains in the research index.

Corrective RAG grades retrieval with a lightweight T5 evaluator and then Correct-refines, Incorrect-falls back to the web, or Ambiguous-blends before generation (Yan et al., 2024). Published PopQA generation overhead rises from 0.363 s to 0.512 s with Table 6’s caveat that web and retrieval stages sit outside that figure. Use the pattern when bad retrieval reaches the generator and you can maintain an evaluator; skip it when web fallback violates perimeter rules or when reflection-token Self-RAG is the control loop you actually need. Measure accuracy and p95 on your own labelled set before treating the paper deltas as a production SLA.