Measuring Retrieval: Recall, Precision, MRR and NDCG

Retrieval metrics score the ranked chunk list before generation — whether the right passage was fetched, and how high it ranked. They cannot grade the written answer. This page works Precision@k, Recall@k, MRR and NDCG on one real query list, then says which meter to read for which job.

What do retrieval metrics measure?

Retrieval metrics measure whether the right chunks appear in the top-k and how they are ordered — nothing about the generated answer. Every score needs a labelled set of relevant passages for each query (human labels or a trusted gold set). In RAG, k is usually the number of chunks you actually put in the model’s context window, so the metric grades the same cut-off the generator sees. The broader family list sits on the metrics catalogue; building the labelled set is golden test sets.

How do you calculate Precision@k and Recall@k?

Precision@k is the fraction of the top-k results that are relevant. Recall@k is the fraction of all known relevant items that appear anywhere in that top-k. Hit@k is 1 if at least one relevant item appears, else 0 — a binary recall floor.

LangCopilot’s single-query example (2025) is the list most RAG metric posts reuse. Relevant ground truth: {doc3, doc5, doc10}. Top-5 retrieved: [doc2, doc5, doc3, doc8, doc7]. Recomputed here:

High recall with low precision means the window is noisy — the right chunk may be present beside junk (wrong chunk). High precision with low recall means a clean window that still missed evidence (missing document). Average these metrics over many queries; a single query is for learning the arithmetic.

How does MRR score a ranking?

Mean Reciprocal Rank (MRR) is the mean, over queries, of one over the rank of the first relevant item (Wikipedia, Mean reciprocal rank; Evidently AI, MRR guide). On the same list, the first relevant hit is doc5 at rank 2, so the reciprocal rank is 1÷2 = 0.50 — matching LangCopilot’s expected MRR for this data.

MRR’s blind spot: it ignores every relevant item after the first. A ranking that puts one correct chunk at rank 1 and noise below still scores 1.0 for that query, even when the rest of the window is useless (FutureAGI, 2026 — their Retriever A vs B comparison). Use MRR when the application only consumes the top hit; do not use it alone for multi-chunk RAG context.

How does NDCG score a ranking?

Normalised Discounted Cumulative Gain (NDCG) scores the whole top-k with a log position discount, then divides by the score an ideal ranking would get (IDCG). Relevant hits lower in the list count less; graded labels (partially relevant) are allowed when you have them.

LangCopilot’s post stops at Precision, Recall and MRR for this list. Extending it with binary NDCG@5 (gain = 2^rel−1, standard DCG): relevances for the five ranks are [0, 1, 1, 0, 0].

• DCG = (2¹−1)/log₂(3) + (2¹−1)/log₂(4) = 1.1309
• IDCG (three relevant docs, ideal top-5 = [1, 1, 1, 0, 0]) = 2.1309
• NDCG@5 = 1.1309 ÷ 2.1309 = 0.53 (computed for this page, 2026-07-27)

Use NDCG when order inside the window matters — especially after reranking — or when judges give graded relevance instead of a binary yes/no (Towards Data Science, DCG@k and NDCG@k; Weaviate, retrieval evaluation metrics, 2023).

Which retrieval metric should you use?

Pick the metric that matches what your application actually reads from the ranked list (FutureAGI, MRR vs MAP vs NDCG, 2026):

• Recall@k — default RAG sanity check: did the answer ever enter the context window?
• Precision@k — is that window mostly on-topic, or drowning the model in noise?
• MRR — top-1 / first-hit jobs (for example agent tool routing), not multi-chunk chat context alone.
• NDCG — graded relevance, or when mid-list order changes what the model uses.
• MAP — binary multi-relevant corpus search; average precision across ranks. Named here only — see the catalogue for where it sits in the panel.

For a typical RAG chat stack: start with Recall@k and Precision@k on a labelled set; add NDCG when you introduce a reranker or graded labels. Hybrid lexical + dense retrieval changes these scores — measure both sides on hybrid retrieval.

What does a low retrieval score mean?

A low retrieval score points at a retrieval failure, not a generator failure — unless every retrieval meter is fine and the answer is still wrong.

• Low Recall@k or Hit@k — the relevant passage never made the window → missing document.
• Low Precision@k — top-k is mostly the wrong neighbours → wrong chunk.
• OK recall, low MRR or NDCG — the right doc is present but buried where the model under-weights it → still a ranking / wrong-chunk problem; check lost in the middle once generation sees a long context.
• Retrieval metrics high, answers still wrong — leave this page: check stale index and generation metrics.