BM25 Explained for RAG Engineers

BM25 is a lexical ranking function: it scores a document against a query from three signals only — how often the query’s terms appear in the document (term frequency), how rare those terms are across the whole collection (inverse document frequency), and how long the document is (length normalisation). It reads no meaning. It matches strings. In a modern RAG retrieval stack it is the retriever you keep for the queries embeddings get wrong: exact terms, identifiers, error codes, and rare tokens.

Is BM25 the same as embeddings or TF-IDF?

Most confusion about BM25 comes from assuming it does something it does not. It is defined here by its borders.

How does BM25 score a document?

For a query Q and document D, BM25 sums one contribution per query term. Each term’s contribution is its inverse document frequency multiplied by a saturating term-frequency factor that is damped when the document is longer than average:

score(D, Q) = Σ  IDF(qᵢ) ·  f(qᵢ,D)·(k₁+1)
                          ─────────────────────────────────
                          f(qᵢ,D) + k₁·(1 − b + b·|D|/avgdl)

  f(qᵢ,D)  term qᵢ's frequency in D        k₁ = 1.5   (TF saturation)
  |D|      length of D in tokens           b  = 0.75  (length normalisation)
  avgdl    average document length         IDF(qᵢ) = ln( (N − n(qᵢ) + 0.5) / (n(qᵢ) + 0.5) + 1 )

The two constants have jobs you can feel. k1 controls how fast term frequency saturates — raise it and repeated terms keep mattering; lower it and one match is nearly as good as five. b controls the length penalty — at b=0 length is ignored, at b=1 it is fully normalised. The defaults k1=1.5, b=0.75 are the values almost every engine ships (Elastic’s Practical BM25 walks through what happens as you move them). Tuning k1 and b against your own corpus is worth doing, but it is a production step, not part of the definition — do it where you assemble retrieval.

A worked example, scored end to end

Take a five-document collection and the query “error code 502”. With N=5, avgdl=7.6 tokens, the term statistics give IDF(error)=0.539, IDF(code)=0.875, IDF(502)=0.875 — “error” scores lower because it appears in more documents, so it is less discriminating. Scoring every document:

D1 wins decisively — it is the only document that contains all three terms. But look at rank 2: D4 outranks D5 purely because it repeats the common word “code,” even though it is about code review and has nothing to do with a 502 error. That is BM25 being honest about what it is: a string counter, not a reader. It is also exactly why you pair it with a dense retriever rather than replacing one with the other.

What are BM25’s limitations?

Because BM25 only counts strings, it is blind in four specific ways — and each blind spot is exactly what dense retrieval is good at, which is why you end up running both rather than choosing one.

• Synonym blindness. “car” and “automobile,” “502” and “bad gateway” share no tokens, so BM25 scores them as unrelated. The meaning is identical; the strings are not.
• Common terms dominate short documents. A short doc that happens to repeat a frequent query word can outrank a longer, genuinely more relevant one — length normalisation dampens this but does not remove it.
• Repeated keywords can game the score. Term-frequency saturation (the k1 cap) limits keyword stuffing, but a document engineered to repeat the query terms still climbs.
• Stop-word loss. Words filtered as noise are sometimes the signal — “to be or not to be” is almost entirely stop words, and BM25 keeps little of it.

None of these are reasons to abandon BM25; they are the precise cases where you add an embedding retriever alongside it, so the two cover each other’s blind spots.

BM25 vs embeddings: when does BM25 win?

Embeddings map text into a space where nearby vectors mean similar things — which is precisely the wrong tool when the surface form is the signal. BM25 wins whenever the exact token matters more than its meaning:

• Identifiers and codes — SKU-4471, CVE-2024-3094, HTTP 502. An embedding smears these toward “similar-looking” IDs; BM25 demands the literal match.
• Exact names and rare tokens — a surname, a library version, a config key. Rare terms get the highest IDF, so BM25 weights them heavily; embeddings under-represent tokens they saw little of in training.
• Quoted phrases the user expects verbatim — error strings copy-pasted from a log, a legal clause, a function signature.

This is the concrete failure practitioners hit — a dense retriever that “doesn’t give expected accuracy” on exact-term queries. The fix is not a better embedding model; it is adding the lexical signal back.

Where does BM25 fit in hybrid retrieval?

You rarely choose BM25 or embeddings. You run both and fuse the results — dense retrieval for meaning, BM25 for exact terms — which is hybrid search. This page defines the lexical half; the hybrid page shows how to combine them and weight the fusion.