RAG Glossary: Every Term, Defined Once

Every term below is defined once, in one or two sentences, and links to the page that explains it in full. If a term isn’t here, it’s because no page explains it yet — this glossary never defines a term it can’t route you to.

Core concepts

RAG (retrieval-augmented generation) — Answering a question by fetching relevant passages from your own documents at query time and giving them to a language model, so the answer is grounded in your data rather than the model’s training weights.

Retrieval — The stage that finds and ranks the passages most relevant to a query. In RAG, retrieval quality sets the ceiling on answer quality: the model can only use what retrieval returns.

Chunking — Splitting source documents into passages small enough to embed and retrieve. How you split — by length, structure, or meaning — decides whether an answer lands inside one chunk or gets cut across a boundary.

Embeddings — Dense numeric vectors that place text in a space where nearby vectors mean similar things, so a query can be matched to passages by meaning rather than exact words.

Vector search — Finding the passages whose embeddings are nearest to the query’s embedding, typically by cosine similarity. The dense half of retrieval; contrast with lexical BM25.

Generation — The final stage, where a language model writes the answer from the query and the retrieved context. Good generation stays faithful to that context and doesn’t invent beyond it.

Grounding — Tying every claim in the answer to the retrieved context, so the model quotes evidence rather than its own priors. The mechanism that turns retrieval into a trustworthy answer.

Context window — The maximum amount of text a model can read at once. It bounds how many retrieved chunks fit in the prompt, which is why retrieval must rank, not just fetch.

Fine-tuning — Continuing to train a model on your data to change its behaviour. The contrast to RAG: fine-tuning changes how the model acts; RAG changes what it knows at answer time.

Retrieval mechanics

BM25 — A lexical ranking function that scores a document by term frequency, inverse document frequency, and length. It matches exact strings, not meaning, and still beats embeddings on identifiers, codes and rare tokens.

Hybrid search — Running dense (embedding) and sparse (BM25) retrieval together and fusing the results, so a query is matched on both meaning and exact terms.

Reranking — A second pass that re-orders the top retrieved candidates with a more accurate but slower model, promoting the truly relevant chunk above merely similar ones.

Cross-encoder — A reranking model that reads the query and a candidate passage together and scores their relevance directly. More accurate than embedding similarity, too slow to run over the whole index — so it reranks a shortlist.

Top-k — The number of highest-ranked chunks retrieval returns to the model. Too small and the answer may be cut off; too large and it adds noise and cost.

Semantic search — Retrieval by meaning rather than keyword overlap, powered by embeddings. Another name for the dense side of vector search.

Cosine similarity — The measure of how close two embeddings point in the same direction, ignoring their length. The usual score for ranking passages against a query in vector search.

Sparse embeddings — Vectors with mostly zero entries that encode which terms matter, bridging keyword and dense retrieval. The learned middle ground between BM25 and dense embeddings.

Query rewriting — Reformulating the user’s question before retrieval — expanding, clarifying or splitting it — so it better matches how the answer is phrased in the documents.

HyDE — Hypothetical Document Embeddings: generate a fake answer to the query, embed that, and retrieve against it — because a hypothetical answer sits closer to the real passages than the question does.

ColBERT — A late-interaction retriever that scores each query token by its best-matching document token (MaxSim), catching term-level matches a single pooled embedding blurs. More precise than dense similarity, heavier on storage.

Indexing & infrastructure

Vector database — A store built to index embeddings and return the nearest ones to a query fast. The component that makes retrieval by meaning practical at scale.

HNSW — Hierarchical Navigable Small World: the graph-based index most vector databases use for approximate nearest-neighbour search. Fast and accurate, but it must be held in memory.

Quantization — Compressing vectors to fewer bits so more fit in memory, trading a little retrieval accuracy for a large cut in RAM. How large vector indexes stay affordable.

Knowledge graph — A store of entities and the explicit relationships between them. Retrieved over instead of, or alongside, a flat passage index — the substrate of Graph RAG.

Evaluation

Faithfulness — Whether the generated answer is actually supported by the retrieved context, rather than invented. The metric that catches hallucination.

Context precision — Of the chunks retrieved, how many were actually relevant. Low precision means retrieval is returning noise alongside the answer.

Context recall — Of the chunks needed to answer, how many retrieval actually found. Low recall means the answer’s evidence never reached the model.

Answer relevance — Whether the generated answer actually addresses the question asked, independent of whether it’s faithful to the context.

Ragas — An open-source framework that scores RAG systems on faithfulness, context precision and recall, and answer relevance, largely using an LLM as the judge.

nDCG — Normalised discounted cumulative gain: a ranking metric that rewards putting the most relevant chunks highest, discounting relevance found further down the list.

Recall@k — The fraction of relevant chunks that appear in the top-k retrieved results. The workhorse metric for measuring whether retrieval found the answer at all.

LLM-as-a-judge — Using a language model to score another model’s answers for qualities like faithfulness or relevance. Scalable, but it inherits the judge’s biases and varies run to run — so you average, not trust a single pass.

Failure terms

Hallucination — When the model generates a confident answer not supported by the retrieved context. In RAG it usually signals a generation failure, or context that never contained the answer.

Lost in the middle — The tendency of a language model to under-use information placed in the middle of a long context, even when the right chunk was retrieved.

Chunk boundary — The split point between two chunks. When an answer straddles a boundary, no single chunk contains it whole and retrieval returns half the answer.

Drift — Gradual degradation of retrieval quality as the corpus or the query distribution shifts away from what the system was built and tuned for.

Leakage — Returning documents a user should not see — another tenant’s or another permission scope’s — usually because a metadata filter is missing at query time.

Architecture terms

Naive RAG — The baseline shape: retrieve top-k by embedding similarity, stuff the chunks into the prompt, generate. Simple, and the starting point every failure mode is measured against.

Advanced RAG — Naive RAG plus retrieval improvements — hybrid search, reranking, query rewriting — that raise precision and recall before generation.

Agentic RAG — An architecture where the model plans and issues multiple retrieval steps, deciding what to fetch next based on what it has found so far.

Graph RAG — Retrieval over a knowledge graph rather than a flat passage index, so answers can traverse explicit relationships between entities.

Multi-hop — A question that can only be answered by chaining two or more facts, often across separate documents. A known weakness of single-shot retrieval.

Modular RAG — Treating the pipeline as swappable modules — rewriter, retriever, reranker, generator — reconfigured per use case rather than one fixed chain. The framing the advanced variants below are instances of.

RAPTOR — An architecture that clusters and summarises chunks into a tree, so retrieval can pull a high-level summary or a leaf detail depending on how broad the question is.

Self-RAG — A model trained to decide when to retrieve and to critique its own retrieved context and output, retrieving only when it helps rather than on every query.

Corrective RAG — Grades retrieved documents for relevance and, when they fall short, triggers a corrective step such as a web search before generating. Retrieval with a quality gate.

CAG (cache-augmented generation) — Preloading the whole corpus into a long context window and caching it, skipping retrieval entirely. Viable only when the corpus fits the window — the contrast that shows what retrieval is for.