Which Reranker Should You Use?

The practical reranker pick in 2026 is a shortlist, not a single winner: Cohere and Voyage for hosted APIs, Jina and BGE for self-hosted open weights, and Nemotron when Hit@1 headroom beats a sub-200 ms slice. Machine Learning Mastery’s September 2026 top-five list and AIMultiple’s reranker benchmark (captured 17 September 2026) agree on that field while disagreeing on rank order, which is why this survey sorts by published Hit@1 where a shared pipeline exists and otherwise by deploy model. Weaviate, Qdrant, and Pinecone can call those models after search; they do not replace the model choice. Parent gate for whether you need a second stage at all sits on reranking in RAG.

What does a reranker model actually score?

A reranker scores each query-document pair jointly and reorders the shortlist that retrieval already returned , it cannot invent a document the retriever never fetched. The two-stage pattern is the same across the top-ranking results for best reranker models (Agentset, Bswen, AIMultiple, Particula, Analytics Vidhya , captured 2026-07-27): a fast retriever casts a wide net for recall, then a slower cross-encoder rescores the shortlist for precision.

Galileo’s selection guide (captured 17 September 2026) splits the field into open cross-encoders such as BGE, private rerank APIs such as Cohere and Jina, and late-interaction options such as ColBERT. Mechanism depth for why pairwise scoring beats bi-encoder similarity is why cross-encoders beat retrieval alone.

What that means for the comparison below:

• Pair scoring , the model reads the query and one candidate together and emits a relevance score (the usual cross-encoder). Why that beats compressed bi-encoder similarity is cross-encoders explained.
• Retriever ceiling , on AIMultiple’s English Amazon-reviews set (Sarı, updated 2026-07-15), every top reranker converged around 87-88% Hit@10 because multilingual-e5-base never placed the right document in the top-100 candidates for the remaining queries. No reranker recovers a miss that early.
• Hit@1 and latency class , Hit@1 asks whether the top result is correct; latency class is the added inference (and, for APIs, network) cost per query. Those are the axes the table uses.

Whether you need a reranker at all , high recall@50 with low recall@10 , is the gate on reranking in RAG. The bake-off below assumes that gate already passed.

How do Cohere, Voyage, Jina, BGE and Nemotron compare?

On AIMultiple’s shared English Amazon-reviews pipeline (retrieve top-100 with multilingual-e5-base, rerank, evaluate; captured 17 September 2026), Hit@1 order is Nemotron at 83.00%, then Jina v3 at 81.33%, with the no-rerank baseline at 62.67%. That is a scored table sorted by Hit@1 on one corpus, not a universal ranking. Hosted Cohere and Voyage are compared on deploy model and latency class because that benchmark did not publish matching Hit@1 for every hosted API in the same run.

The five at a glance, then a short honest profile of each , strength and limitation both kept in. Numbers are published third-party figures, not this site’s own leaderboard; treat them as directional and re-measure on your corpus.

Rank numerals here mark reading order, not a verdict , the right choice depends on your situation, which is two sections down. On AIMultiple’s same pipeline (retrieve top-100 with multilingual-e5-base, rerank, evaluate top-10 on 300 queries), the best reranker lifted Hit@1 from 62.67% to 83.00% (+20.33 percentage points). That is AIMultiple’s figure on Amazon reviews , not a universal constant. For the latency math that decides whether that gain pays for itself, see what reranking costs you in latency.

Should you use a hosted reranker API or self-host?

Hosted APIs (Cohere, Voyage) win when you have no GPU and moderate traffic; self-host (Jina, BGE, Nemotron) wins when you need sub-200 ms wire latency, high QPS, or data residency. Particula’s published contrast (May 2026) is the cleanest statement of the trade: hosted Cohere-class calls sit around 595-603 ms including network round-trip, while a local Jina v3 run on their cited AIMultiple numbers is 188 ms and Nemotron 243 ms.

How the two sides differ in practice:

• Hosted , one API call, no model serving, per-call pricing that scales with traffic. Time-to-ship is minutes. The cost you feel first is round-trip latency, not dollars.
• Self-host , you take GPUs, batching and monitoring; you gain wire latency, no per-call fee, and full control over where documents are scored. Bswen’s BGE timings show why hardware matters: ~350 ms on CPU vs ~80 ms on a T4 for three documents.

Modules that call a reranker from a vector store: Weaviate, Qdrant, and Pinecone , are wiring. They plug the model in; they do not choose which model is right. The break-even against candidate-set size lives on reranking cost and latency.

Which reranker should you choose?

Match the model to your situation, not to a leaderboard:

Match the model to your constraints after the when-to gate on the reranking hub has passed. Weaviate collections can call Cohere, Voyage AI, or a Transformers cross-encoder through the documented reranker modules (as of September 2026); that wiring is independent of which model wins on your labelled set.

• Want a hosted default on a general corpus, no GPU → Cohere Rerank.
• Corpus is source code or legal text → test Voyage’s domain variant first (Particula’s +2-4 NDCG@10 claim is on-domain only).
• Need a strict sub-200 ms rerank slice and a GPU → Jina Reranker v3 (check the CC BY-NC license fits your use).
• Want max Hit@1 with latency headroom → Nemotron rerank 1B v2 (or the smaller gte-reranker-modernbert-base, which tied it at 83.00% Hit@1 on AIMultiple’s set).
• Need multilingual on a budget → BGE-reranker-v2-m3.
• English-only CPU prototype → ms-marco-MiniLM-L-6-v2 or FlashRank can sit under ~50 ms (Bswen / Analytics Vidhya), but they are not the production default once accuracy matters.

If you only adopt one open-source baseline to beat, start with BGE-reranker-v2-m3: it is the practical floor Machine Learning Mastery still recommends in 2026 , if a newer model does not clearly beat it on your labelled set, the extra cost is not justified. Why pairwise scoring is worth that second stage at all is cross-encoders explained.

What do teams get wrong when picking a reranker?

The model pick is rarely the hard part , measuring on your corpus is. Five recurring mistakes:

Leaderboard transfer is the other failure: AIMultiple’s +20.33 percentage-point Hit@1 lift is on Amazon reviews with a fixed retriever. Your corpus, chunker, and first-stage recall change the ceiling. Measure on your labelled queries with retrieval metrics before you declare a winner.

• Crowning a vendor leaderboard. A page that sells Zerank and ranks Zerank first is not an independent survey. Re-weight by your constraints, or use a publication that sells none of the models.
• Assuming any reranker helps. AIMultiple found mxbai-rerank-xsmall at 64.67% Hit@1 against a 62.67% no-reranker baseline , a ~2 percentage-point lift inside noise for 300 queries. Test before you deploy.
• Unbounded candidate counts. Rerank latency scales with how many documents you score. Particula’s production note: cap first-stage top-k around 50-100 before the reranker; feeding 500 candidates is the common P95 surprise.
• Judging on average latency. Report P95. Long documents and fat candidate sets hide in the tail while the mean still looks fine.
• Expecting recovery of retrieval misses. If the right chunk never enters the candidate set, no cross-encoder invents it , fix retrieval (hybrid search, chunking) first, then rerank.

How to score the lift with Hit@k / nDCG / MRR on a labelled set is at retrieval metrics; when the added milliseconds stop paying for the quality gain is at reranking cost.

Machine Learning Mastery’s 2026 shortlist still treats BGE-reranker-v2-m3 as the practical open baseline and Cohere rerank-v4.0-pro as the managed option to test first. Redis adds that mid-size cross-encoders usually cost tens to low hundreds of milliseconds, while LLM rerankers can take seconds. Galileo’s checklist is evaluation-first: measure ranking lift, then latency, then cost.

Weaviate’s documentation (as of September 2026) lists Cohere, Voyage AI, and Transformers reranker integrations as collection configuration. That is wiring after the pick, not a claim that any hosted model is best. Score on your labelled set; sort published bake-offs by the metric they actually share.

Run the bake-off on your corpus once; do not promote a leaderboard winner that was never measured against your chunker and retriever.

Hosted APIs win on time-to-ship; self-hosted weights win when wire latency or residency dominates the constraint list.

Particula and Agentset latency snapshots for hosted Cohere-class calls sit near 600 milliseconds including network round-trip on their published figures; treat those as directional and re-measure in your region.

Self-hosted Jina v3 clearing a sub-200 millisecond slice on AIMultiple’s 100-candidate H100 run is a hardware-specific result, not a promise for CPU-only deployments.

Pick once on your labelled set, then freeze the winner until the retriever or chunker changes.

How do you wire a reranker through Weaviate?

Weaviate wires a reranker as a collection-level model-provider integration, then applies it after the initial search on each query. The Cohere integration docs (as of September 2026) require the reranker-cohere module, accept models such as rerank-english-v3.0, and note a server default of rerank-v3.5 on recent versions. The Voyage AI integration uses reranker-voyageai with models including rerank-2.5, rerank-2.5-lite, and rerank-lite-1 as the documented default when unspecified.

The Transformers integration runs a local inference container tagged for a cross-encoder such as cross-encoder-ms-marco-MiniLM-L-6-v2. In all three cases the query path is the same: search first, then pass a Rerank argument naming the property and the query string so Weaviate reorders hits before they leave the API. That wiring does not choose the winner on your labelled set; it only makes the second stage callable next to hybrid or vector search in Weaviate, Qdrant, or Pinecone pipelines that expose a similar hook.

Configuration mutability for reranker settings is documented from Weaviate v1.26.8 and v1.27.1 onward for those modules. Treat version pins as volatile: check the current model-provider page before you ship, and keep the as-of date in your runbook beside the model ID you actually call.

There is no single best reranker for every corpus. AIMultiple’s Hit@1 order, Machine Learning Mastery’s 2026 shortlist, and Weaviate’s Cohere, Voyage, and Transformers modules answer different questions: score on your data, pick hosted versus self-host from the latency budget, then wire the winner through the store you already run. Latency break-evens live on reranking cost and latency; late interaction and LLM listwise paths are separate leaves under reranking.

More in Reranking in RAG: The Cheapest Large Quality Gain Toggle section list