HyDE: Retrieving With a Hypothetical Answer

HyDE (Hypothetical Document Embeddings) generates a fake answer to the query, embeds that passage, and retrieves real documents by similarity to the hypothetical — not to the raw question. The invented details are thrown away after retrieval; only real chunks reach the generator. The method comes from Gao, Ma, Lin and Callan (2022; arXiv:2212.10496), who designed it for zero-shot dense retrieval when no relevance labels exist. It attacks the same vocabulary mismatch that makes short questions sit far from long answers in embedding space — and it backfires when latency is tight or the fake answer drifts into the wrong domain.

How does HyDE retrieval work?

HyDE changes what gets embedded at query time. The index is unchanged; the query vector is built from a generated passage instead of from the question:

1. The query arrives. The user asks a natural-language question — often short and under-specified.
2. An instruction-following LLM drafts a hypothetical document. The model is prompted to write a passage that would answer the question (Gao et al., 2022). That draft may contain false details; that is expected.
3. A dense encoder embeds the hypothetical. The original paper uses an unsupervised contrastive encoder (Contriever) so the search vector lives in the same space as the corpus embeddings.
4. Vector search retrieves real documents. Nearest neighbours of the hypothetical vector are returned from the index. The hypothetical is discarded; generation runs only on those real chunks.

Baseline dense retrieval embeds the question and compares question-shaped vectors to answer-shaped chunks. HyDE compares document-shaped text to document-shaped text. Gao et al. (2022 §4.1) built the reference stack with InstructGPT (text-davinci-003) for generation and Contriever for encoding — modern pipelines swap both models but keep the same four steps.

Why can a wrong hypothetical still retrieve the right document?

A wrong hypothetical can still retrieve the right document because HyDE uses the draft as a retrieval vector, not as trusted knowledge. Gao et al. (2022) state the generated document “captures relevance patterns but is unreal and may contain false details,” then rely on the encoder’s dense bottleneck to filter incorrect specifics and ground the vector in the neighbourhood of real corpus passages.

The geometry is the same one freeCodeCamp’s HyDE guide (2026) emphasises: embedding models place similar shapes of text near each other. A question and its answer are different shapes; a fake answer and a real answer are the same shape, so they share vocabulary, register and length even when individual facts differ. The metric that matters is whether the retrieved chunks are correct — not whether the scaffolding paragraph was.

How much does HyDE improve retrieval?

On the web-search sets in Gao et al. (2022, Table 1), HyDE lifts unsupervised Contriever by a large margin and sits close to a Contriever fine-tuned on MS MARCO — without using relevance labels. The published nDCG@10 and recall@1k figures are:

On DL19, HyDE’s nDCG@10 of 61.3 is within 0.8 of Contriever^FT’s 62.1, and HyDE posts the best recall@1k in that table (88.0). On DL20, HyDE’s nDCG@10 of 57.9 trails Contriever^FT’s 63.2 — the paper describes map and nDCG as about 10% lower there, with similar recall@1k. Those numbers are for that InstructGPT + Contriever stack on those benchmarks as of the 2022 paper; they are not a promise for your corpus. How to read nDCG and recall in a RAG eval is at retrieval metrics.

What does HyDE cost at query time?

HyDE’s structural cost is one extra LLM generation call per query before retrieval, plus one embedding of the hypothetical. Baseline dense retrieval makes zero generation calls at retrieve time; HyDE pays that generation cost on every query, not once at ingest.

• Latency scales with the generation model. The added wall-clock time is whatever that draft call takes on your stack — measure it; do not trust a blog’s millisecond claim.
• Token spend is paid twice when you also generate the final answer. One call drafts the hypothetical; a later call answers from real chunks. Caching identical queries and using a small, fast model for the draft are the usual mitigations (freeCodeCamp 2026; SurePrompts 2026).
• Cap draft length. Over-long hypotheticals dilute the embedding. Keep the prompt capped to roughly a paragraph (or your median chunk length) so the search vector stays dense.
• Fail open. If generation times out or errors, fall back to embedding the raw query rather than blocking the request.

When should you use HyDE?

Use HyDE when the bottleneck is query–document asymmetry and you can afford a generation call on the retrieval path. Skip it when a cheaper channel already closes the gap — or when the draft would pull search the wrong way.

• Use it for short, under-specified questions over prose-heavy corpora (docs, papers, memos) where users speak lay language and documents speak expert register — and when you lack labelled query–document pairs to fine-tune a dense retriever (HyDE’s zero-shot setting in Gao et al., 2022).
• Skip it when queries are already long and specific, when they are identifier-heavy (error codes, SKUs, statute numbers — prefer hybrid search / BM25), when latency budgets forbid an extra LLM hop, or when the domain is so alien that a general model invents jargon that does not exist in your corpus (hallucinated drift).
• Do not confuse it with query rewriting. Query rewriting cleans or expands a question while it remains a question. HyDE changes the shape of the search text into a document-register passage. They compose; they are not substitutes. SurePrompts (2026) places HyDE after chunking, hybrid and rewrite in the usual adoption order — measure recall@k on a golden set before making HyDE the default.

How do you implement HyDE?

The implementation pattern is short: prompt an LLM for a document-register draft, embed that draft with the same embedding model your index already uses, run ANN search, discard the draft, then generate from the retrieved chunks. Frameworks expose the pattern as LangChain’s HyDE / HypotheticalDocumentEmbedder path and LlamaIndex’s HyDEQueryTransform (verify the current import path as of July 2026 — APIs move). The vector store that holds the dense index — Weaviate, Qdrant, Milvus, and others — is interchangeable for the search step; HyDE changes what is embedded, not which database you use.

Runnable, pinned end-to-end code belongs with building the pipeline. Prove whether HyDE earns its latency on retrieval metrics for your own queries — the Gao table is the literature baseline, not your production score.