Which Reranker Should You Use?

Five models cover the real pick: Cohere Rerank (hosted), Voyage Rerank (hosted), Jina Reranker v3 (self-host, long-context), BGE-reranker-v2-m3 (multilingual open-source baseline), and NVIDIA llama-nemotron-rerank-1b-v2 (top Hit@1 on a published English benchmark). There is no single best reranker — match deploy model, latency budget and corpus, then measure on your own labelled set.

What does a reranker model actually score?

A reranker scores each query–document pair jointly and reorders the shortlist that retrieval already returned — it cannot invent a document the retriever never fetched. The two-stage pattern is the same across the top-ranking results for best reranker models (Agentset, Bswen, AIMultiple, Particula, Analytics Vidhya — captured 2026-07-27): a fast retriever casts a wide net for recall, then a slower cross-encoder rescores the shortlist for precision.

What that means for the comparison below:

• Pair scoring — the model reads the query and one candidate together and emits a relevance score (the usual cross-encoder). Why that beats compressed bi-encoder similarity is cross-encoders explained.
• Retriever ceiling — on AIMultiple’s English Amazon-reviews set (Sarı, updated 2026-07-15), every top reranker converged around 87–88% Hit@10 because multilingual-e5-base never placed the right document in the top-100 candidates for the remaining queries. No reranker recovers a miss that early.
• Hit@1 and latency class — Hit@1 asks whether the top result is correct; latency class is the added inference (and, for APIs, network) cost per query. Those are the axes the table uses.

Whether you need a reranker at all — high recall@50 with low recall@10 — is the gate on reranking in RAG. This page assumes that gate already passed.

How do Cohere, Voyage, Jina, BGE and Nemotron compare?

The five at a glance, then a short honest profile of each — strength and limitation both kept in. Numbers are published third-party figures, not this site’s own leaderboard; treat them as directional and re-measure on your corpus.

Rank numerals here mark reading order, not a verdict — the right choice depends on your situation, which is two sections down. On AIMultiple’s same pipeline (retrieve top-100 with multilingual-e5-base, rerank, evaluate top-10 on 300 queries), the best reranker lifted Hit@1 from 62.67% to 83.00% (+20.33 percentage points). That is AIMultiple’s figure on Amazon reviews — not a universal constant. For the latency math that decides whether that gain pays for itself, see what reranking costs you in latency.

Should you use a hosted reranker API or self-host?

Hosted APIs (Cohere, Voyage) win when you have no GPU and moderate traffic; self-host (Jina, BGE, Nemotron) wins when you need sub-200 ms wire latency, high QPS, or data residency. Particula’s published contrast (May 2026) is the cleanest statement of the trade: hosted Cohere-class calls sit around 595–603 ms including network round-trip, while a local Jina v3 run on their cited AIMultiple numbers is 188 ms and Nemotron 243 ms.

How the two sides differ in practice:

• Hosted — one API call, no model serving, per-call pricing that scales with traffic. Time-to-ship is minutes. The cost you feel first is round-trip latency, not dollars.
• Self-host — you take GPUs, batching and monitoring; you gain wire latency, no per-call fee, and full control over where documents are scored. Bswen’s BGE timings show why hardware matters: ~350 ms on CPU vs ~80 ms on a T4 for three documents.

Modules that call a reranker from a vector store — Weaviate, Pinecone, Qdrant — are wiring. They plug the model in; they do not choose which model is right. The break-even against candidate-set size lives on reranking cost and latency.

Which reranker should you choose?

Match the model to your situation, not to a leaderboard:

• Want a hosted default on a general corpus, no GPU → Cohere Rerank.
• Corpus is source code or legal text → test Voyage’s domain variant first (Particula’s +2–4 NDCG@10 claim is on-domain only).
• Need a strict sub-200 ms rerank slice and a GPU → Jina Reranker v3 (check the CC BY-NC license fits your use).
• Want max Hit@1 with latency headroom → Nemotron rerank 1B v2 (or the smaller gte-reranker-modernbert-base, which tied it at 83.00% Hit@1 on AIMultiple’s set).
• Need multilingual on a budget → BGE-reranker-v2-m3.
• English-only CPU prototype → ms-marco-MiniLM-L-6-v2 or FlashRank can sit under ~50 ms (Bswen / Analytics Vidhya), but they are not the production default once accuracy matters.

If you only adopt one open-source baseline to beat, start with BGE-reranker-v2-m3: it is the practical floor Machine Learning Mastery still recommends in 2026 — if a newer model does not clearly beat it on your labelled set, the extra cost is not justified. Why pairwise scoring is worth that second stage at all is cross-encoders explained.

What do teams get wrong when picking a reranker?

The model pick is rarely the hard part — measuring on your corpus is. Five recurring mistakes:

• Crowning a vendor leaderboard. A page that sells Zerank and ranks Zerank first is not an independent survey. Re-weight by your constraints, or use a publication that sells none of the models.
• Assuming any reranker helps. AIMultiple found mxbai-rerank-xsmall at 64.67% Hit@1 against a 62.67% no-reranker baseline — a ~2 percentage-point lift inside noise for 300 queries. Test before you deploy.
• Unbounded candidate counts. Rerank latency scales with how many documents you score. Particula’s production note: cap first-stage top-k around 50–100 before the reranker; feeding 500 candidates is the common P95 surprise.
• Judging on average latency. Report P95. Long documents and fat candidate sets hide in the tail while the mean still looks fine.
• Expecting recovery of retrieval misses. If the right chunk never enters the candidate set, no cross-encoder invents it — fix retrieval (hybrid search, chunking) first, then rerank.

How to score the lift with Hit@k / nDCG / MRR on a labelled set is at retrieval metrics; when the added milliseconds stop paying for the quality gain is at reranking cost.