Hybrid Search: Combining BM25 and Vector Retrieval

Hybrid search runs BM25 and dense vector retrieval in parallel, then fuses both ranked lists so exact tokens and semantic neighbours surface together.

It is the default upgrade when embeddings alone miss identifiers, error codes, and rare terms — and the page that follows measures how fusion works rather than asserting a free recall win.

Why does vector search miss exact terms?

Dense retrieval scores semantic neighbourhood, not string identity. Product codes, error codes, SKUs, statute numbers, and rare tokens often sit in weak or crowded regions of the embedding space, so a semantically adjacent chunk outranks the line that literally contains the identifier the user typed.

A support query like ERR_BLOCKED_BY_CLIENT is the textbook case: the vector index may return generic browser-error prose, while BM25 returns the one document that contains the exact token (TeachMeIdea, hybrid-search guide). Keyword-only search has the opposite blind spot — it cannot bridge paraphrase (“cancel my account” vs “closing your subscription”) — which is why hybrid keeps both channels rather than replacing one with the other.

The detection signature used across this site is recall@k(sparse) − recall@k(dense) > 0 on the failing query: the lexical index can see the gold; the embedding cannot rank it. That failure mode in full is vocabulary mismatch.

How does hybrid search combine BM25 and vectors?

Hybrid search is a three-stage pipeline, not a single score:

1. Dual retrieval. The same query hits a sparse index (BM25 / inverted index) and a dense index (ANN over embeddings) in parallel. Each side returns its own top-k candidate list.
2. Fusion. The two ranked lists become one ranking — usually Reciprocal Rank Fusion (RRF) or a normalised weighted blend (often an alpha between lexical and dense).
3. Optional rerank. A cross-encoder may rescore a shortlist for precision. That stage is owned by reranking, not by fusion.

The reason fusion exists is scale incompatibility. BM25 produces unbounded positive scores shaped by term statistics; cosine similarity typically lives in roughly [-1, 1]. Adding those raw numbers lets whichever channel produces larger magnitudes drown the other — Digital Applied (2026) and Serghei’s RRF walkthrough both treat that as the production gotcha that rank-based fusion was designed to avoid. Stores may keep both signals in one collection (sparse + dense fields) or in two indexes; the architecture rule is the same either way.

How does reciprocal rank fusion work?

Reciprocal Rank Fusion scores a document only by where it ranked in each list — it throws the raw scores away. Cormack, Clarke and Büttcher’s SIGIR 2009 paper defines the sum over retrievers r as:

RRF_score(d) = Σ  1 / (k + rank_r(d))

  rank_r(d)  = 1-based position of d in retriever r’s list
  k          = 60   (paper default; flattens outlier first-place votes)

With k = 60, rank 1 contributes about 0.0164 per list and rank 100 about 0.0063 — so agreement across lists matters more than one loud first-place vote. The same three-document example Serghei (2026) walks through makes that concrete:

At k = 60 the order is B > C > A: B wins because both rankers agreed it was decent. At k = 0 the order flips to C > A — a single first-place vote almost dominates. That is the whole job of k: mitigate outlier systems, as the 2009 paper states. Weaviate exposes the same rank-only mode as rankedFusion beside its default relative score fusion; Qdrant ships Fusion.RRF; Azure AI Search, Elasticsearch, and OpenSearch also fuse hybrid results with RRF.

Is RRF better than weighted score fusion?

RRF is the robust zero-tune default: no score calibration, one constant that is flat across a wide range in the original pilot. Weighted fusion — alpha · dense + (1 − alpha) · sparse after min-max or relative normalisation — is the tunable knob when you have a labelled eval set and want to push lexical or semantic on purpose.

RRF is not automatically the highest NDCG. OpenSearch’s Neural Search team (introducing RRF for hybrid search) measured six BEIR datasets and reported Hybrid+RRF averaging 3.86% lower NDCG@10 than their score-normalised hybrid baseline, with p50 latency about 1.62% better — a robustness/ops trade, not a free win on every corpus. Verify before you treat either default as sacred on your own queries.

Vendor defaults diverge, so pin the fusion type explicitly if you care about reproducibility. As of the Digital Applied (2026) vendor matrix — and consistent with this site’s Weaviate profile — the common production shapes are:

Weaviate’s native hybrid — vector + BM25 in one query, alpha-tunable under relative score fusion — is profiled with its real limits at Weaviate for RAG. Placement in the table does not change the rule: measure fusion choice on your labelled set.

When should you use hybrid search?

Use hybrid search when user queries mix natural language with exact tokens — error codes, product IDs, names, versions, quoted phrases — or when your eval set shows a positive sparse−dense recall gap on those queries.

• Measured lifts exist, and they are corpus-specific. denser.ai’s 2026 hybrid guide summarises Turnbull’s WANDS e-commerce numbers as tuned hybrid NDCG 0.7497 versus BM25 0.6983 and pure vector 0.6953 (about +7.4% versus the best single method). The same guide reports Strich et al. (2026) on financial text+tables: hybrid RRF alone Recall@5 0.695, dense-only 0.587, and hybrid + Cohere rerank 0.816. Treat those as published benchmarks to verify against, not as a promise for your corpus.
• Skip hybrid when the cost buys little. TeachMeIdea’s when-not list is the honest one: small corpora where dense already hits acceptable recall; purely conversational queries with no identifiers; no labelled eval set yet (you cannot tell if you improved); or when standing up a second sparse index would dominate the engineering budget.
• Hybrid cannot invent a synonym the corpus never wrote. If the user says an acronym and every document uses only the expanded form with no shared tokens, BM25 will not bridge the gap — that is query rewriting or an alias dictionary, not more fusion.

Does hybrid search replace reranking?

Hybrid search does not replace reranking. Hybrid raises first-stage recall by covering complementary blind spots; a cross-encoder raises precision by rescoring the shortlist with a query–document pair model. The production pattern is retrieve wide with hybrid → fuse → rerank the top N.

On the financial-document numbers denser.ai attributes to Strich et al. (2026), hybrid + Cohere Rerank reached Recall@5 0.816 against hybrid RRF alone at 0.695 — a large second-stage lift on that set. Skip the reranker only when your hybrid top-k is already clean on the metrics you care about. Depth on the second stage sits at reranking in RAG and cross-encoders.

How do you implement hybrid search in RAG?

Stores with native hybrid — Weaviate, Qdrant, Pinecone, Azure AI Search, Elasticsearch and OpenSearch — remove most glue by accepting both signals in one query path. Otherwise run BM25 and a vector search client-side and fuse with RRF in a short loop (Serghei’s reference implementation is about seven lines). Pin library versions, keep both indexes updated on every write, and score Recall@k / NDCG on a labelled set before and after the change.

This page states the mechanism. The runnable, pinned pipeline lives at building the pipeline; the lexical scorer itself is BM25 explained; the parent map of retrieval choices is retrieval in RAG.