MMR: Diversifying Retrieved Results

Maximal Marginal Relevance (MMR) iteratively selects the next chunk that is relevant to the query and dissimilar to chunks already chosen, so the packed top-k covers more aspects instead of near-duplicate paraphrases. Carbonell and Goldstein introduced the idea for diversity-based reranking in SIGIR 1998 (also summarised in the TIPSTER Phase III workshop paper Goldstein and Carbonell, 1998). In RAG, Elastic’s July 2025 search-labs write-up and Anton Dergunov’s September 2024 Medium walkthrough treat the same relevance-versus-redundancy trade-off that Carbonell and Goldstein formalised. First-stage candidate pools commonly come from Weaviate, OpenSearch, Elasticsearch, and Azure AI Search before a Carbonell-form or product-native diversify step; LlamaIndex and LangChain expose the client-side loop when the store returns embeddings only. Hash deletion of exact copies is the sibling packing step on deduplicating retrieved context; the parent map of retrieval choices is retrieval in RAG.

Why does similarity search return near-duplicate chunks?

Pure similarity top-k ranks by query match alone, so near-paraphrases of the same fact all score high and crowd out a lower-scoring but new aspect. Elastic’s fashion-catalog illustration for the query “pants” shows black-capris variants stacked at cosine-like scores around 0.682, 0.681, and 0.680: each hit is relevant, yet the list repeats one style instead of covering cuts, colours, and brands. Anton Dergunov’s London news-title demo shows the same pattern in text: cosine similarity about 0.710 for “Family-Friendly Activities in London” beside 0.701 for “Family-Friendly Events in London,” and photo-spot titles that paraphrase each other before weather or theatre aspects appear.

Farzzy’s RAG framing names the cost directly: redundant chunks waste the context window and produce answers that look complete in one dimension while missing alternative angles. Chunk overlap as a design cause of near-duplicates lives on chunk size and overlap; deleting exact string clones lives on exact-copy removal; choosing how large final k is lives on choosing retrieval depth k. MMR owns diversity selection among the candidates that remain after first-stage retrieval.

That redundancy problem is why a second score, not a larger k alone, is required: the next pick must pay for similarity to what is already in the prompt.

How does the MMR formula work?

MMR is a greedy selector. It starts with the most query-relevant candidate, then repeatedly scores every remaining candidate with a linear combination of relevance and a penalty for similarity to the closest already-selected item. Elastic, Farzzy, LlamaIndex, and Dergunov all use the Carbonell-form objective common in RAG tooling:

MMR(Di) = λ · Sim(Di, Q) − (1 − λ) · max_j Sim(Di, Dj_selected)

  Sim(Di, Q)              = relevance to the query
  max_j Sim(Di, Dj_sel)   = similarity to the closest already-selected chunk
  λ ∈ [0, 1]              = relevance weight (λ=1 → pure top-k; λ=0 → pure diversity)

LlamaIndex’s notebook writes the same trade-off as an arg-max over the remaining set and names the knob mmr_threshold: values near 1 emphasise relevance; values near 0 emphasise diversity. On Dergunov’s London pool at λ=0.5, the first pick stays the 0.710 family-friendly title, but the next picks open theatre, photo spots, weather, parks, and shopping instead of stacking family-event paraphrases. At λ=0.5 one off-topic Python title also sneaks in; raising λ to 0.7 keeps London-related titles without that drift, while λ=0.8 lets the photo-spot paraphrase pair return.

With the score rule fixed, the open question is which λ bands practitioners actually publish for discovery versus precision workloads.

How do you tune the MMR lambda parameter?

Tune λ by starting relevance-leaning and moving only with measured redundancy; there is no universal published optimum. Elastic’s July 2025 search-labs ranges are the clearest product-facing bands: product discovery λ=0.3–0.5 to emphasise diversity; precision search λ=0.7–0.9 when the user already knows the target; research coverage λ=0.5–0.7; start at λ=0.7 then adjust from user behaviour and result-set similarity. Farzzy’s Azure walkthrough defaults the post-process call to λ=0.7 as a slight relevance bias after retrieving a wide candidate list.

Dergunov quotes the Carbonell and Goldstein guidance that sampling the information space around a query wants a smaller λ (for example λ=0.3), then a reformulated query with a larger λ (for example λ=0.7) to focus. His own London tables make the failure modes visible: λ=0.5 can over-diversify into an off-topic title; λ=0.8 can under-diversify and re-admit paraphrase pairs. OpenSearch’s default diversity=0.5 sits in the middle of their inverted convention, so treat it as a different dial, not a Carbonell starting point.

Measure pairwise similarity inside the packed set and answer coverage on multi-aspect queries before locking a value. The next gate is whether the candidate pool is wide enough for any λ to matter.

What is fetch_k in MMR retrieval?

fetch_k (or OpenSearch candidates) is the candidate pool size retrieved before MMR selects k results. Farzzy’s Azure AI Search example retrieves the top 50 vector hits, then applies MMR down to a top-10 diversified shortlist with λ=0.7. OpenSearch’s candidates default is 3 × the requested size, so a request for ten hits starts from thirty candidates unless you override it.

The hard gate: if the pool is barely larger than k, or every candidate is a near-clone of one aspect, λ cannot invent missing coverage. Dergunov’s weather and theatre titles can only win after the first pick if they are inside the similarity matrix being reranked. Elastic’s performance note is the other side of the same coin: MMR computes similarities between candidates and the growing selected set, so production systems should limit reranking depth to a top-k candidate band and expect vector serialization cost when embeddings must leave the store for client-side scoring.

Framework APIs expose the same split. LangChain’s max_marginal_relevance_search (cited via Dergunov’s LangChain references) takes a final k plus a larger fetch pool and a λ multiplier; LlamaIndex sets MMR through vector_store_query_mode and mmr_threshold. Choosing final packed k without measuring the answer-quality curve still belongs on the top-k answer-quality curve.

Does MMR replace a reranker?

MMR does not replace a cross-encoder reranker. MMR diversifies a first-stage shortlist by penalising similarity to already-selected chunks; a cross-encoder rescores query-document pairs for precision. Farzzy lists semantic reranking beside query rewriting as a complementary lever after MMR, and Elastic notes that relevance into MMR can come from BM25, a learned ranker, or vector similarity, which is not the same as a second-stage pairwise model.

Sreeni Ramadurai’s Dev.to survey places MMR beside self-querying, contextual compression, and ensemble retrieval as separate advanced techniques rather than substitutes for each other. The production pattern that survives is retrieve wide, optionally diversify with MMR, optionally precision-rerank a shortlist. MMR alone does not repair weak first-stage recall. Stage-2 depth lives on reranking and cross-encoders; self-query filters live on self-query retrieval; shrinking text inside a kept chunk lives on contextual compression.

Once the stage boundary is clear, the decision is when the diversity pass is worth the pairwise cost.

When should you use MMR in RAG?

Use MMR when multi-aspect or exploratory queries return near-paraphrase stacks and the context budget is tight. Elastic’s vacation-spots RAG vignette is the archetype: without diversification, “great vacation spots” can collapse to Greek beaches alone; with MMR the shortlist can admit different trip types. Dergunov’s multi-facet London titles and Farzzy’s complex-query framing make the same point for news and enterprise RAG: every slot in the prompt should add an angle, not a clone.

Skip MMR when the corpus is already diverse, when the query is a single precise fact and every top hit is independently necessary, or when the latency budget forbids the extra pairwise pass Elastic warns about. Exact string clones still want hash-based deduplication first: hash deletion frees tokens with no information loss, while MMR keeps k items that are not embedding-space clones. Similarity-metric choice for the Sim terms belongs on similarity metrics, not on the diversity knob.

The honest production pattern is measured: capture pairwise similarity of the packed set before and after MMR, score multi-aspect answer coverage, and keep the pass only where those meters move. LlamaIndex’s notebook makes the score movement concrete: without MMR two nodes can sit near 0.388 and 0.382, while an mmr_threshold of 0.8 or 0.2 rescales and penalises similarity to prior picks so the second node is no longer a near-tie on raw relevance alone.

How do you implement MMR in RAG?

You implement MMR by retrieving a wide candidate pool, running greedy λ-scored selection down to k, then packing. OpenSearch ships a native ext.mmr object (introduced in OpenSearch 3.3 per the docs) with diversity, candidates, and optional explain for per-hit mmr_explain fields. Elastic publishes a Python maximal_marginal_relevance helper against kNN hits (July 2025 search-labs). Farzzy shows the same algorithm as post-processing after Azure AI Search vector retrieval. LlamaIndex demonstrates vector_store_query_mode set to MMR with threshold sweeps that change node scores.

Candidate pools can come from any vector store used for first-stage retrieval, including Weaviate, OpenSearch, Elasticsearch, and Azure AI Search; native MMR extensions are product-specific, so many stacks still run the Carbonell loop client-side on returned embeddings. Pin λ and the candidate multiplier, measure redundancy before and after, and put the runnable pinned end-to-end walkthrough on building the pipeline. Packing after selection returns to context packing after selection and the parent retrieval hub.

Sreeni’s ensemble and compression neighbours are separate nodes: do not treat an MMR notebook as a substitute for hybrid fusion or for extractive compression inside a kept chunk.

MMR is Carbonell and Goldstein’s greedy relevance-versus-diversity selector applied to RAG shortlists so the prompt is not five near-paraphrases of one fact. Use Carbonell-form λ with Elastic and Farzzy starting bands around 0.7 for relevance-leaning work, treat OpenSearch diversity as an inverted dial, keep candidates much larger than final k, and measure pairwise redundancy before you keep the pass. Route hash deletion to dedup, precision rescoring to reranking, and full notebooks to the pipeline build.