HyDE: Retrieving With a Hypothetical Answer

HyDE (Hypothetical Document Embeddings) generates a fake answer to the query, embeds that passage, and retrieves real documents by similarity to the hypothetical, not to the raw question. The invented details are thrown away after retrieval; only real chunks reach the generator. The method comes from Gao, Ma, Lin and Callan (2022; arXiv:2212.10496), who designed it for zero-shot dense retrieval when no relevance labels exist. It attacks the same vocabulary mismatch that makes short questions sit far from long answers in embedding space, and it backfires when latency is tight or the fake answer drifts into the wrong domain. Inside getting the right context to the model, HyDE changes the shape of the search text into document register while leaving the index unchanged.

How does HyDE retrieval work?

HyDE changes what gets embedded at query time. The index is unchanged; the query vector is built from a generated passage instead of from the question. Gao et al. (2022) describe an instruction-following LLM that drafts a hypothetical document answering the question, then an unsupervised dense encoder that embeds that draft so nearest-neighbour search can return real corpus documents.

Baseline dense retrieval embeds the question and compares question-shaped vectors to answer-shaped chunks. HyDE compares document-shaped text to document-shaped text. Gao et al. (2022 §4.1) built the reference stack with InstructGPT (text-davinci-003) for generation and Contriever for encoding; modern pipelines swap both models but keep the same four steps. The hypothetical may contain false details; that is expected for a retrieval scaffold and is not treated as evidence.

1. The query arrives. The user asks a natural-language question, often short and under-specified.
2. An instruction-following LLM drafts a hypothetical document. The model is prompted to write a passage that would answer the question (Gao et al., 2022). That draft may contain false details; that is expected.
3. A dense encoder embeds the hypothetical. The original paper uses an unsupervised contrastive encoder (Contriever) so the search vector lives in the same space as the corpus embeddings.
4. Vector search retrieves real documents. Nearest neighbours of the hypothetical vector are returned from the index. The hypothetical is discarded; generation runs only on those real chunks.

Once the scaffold exists, the surprising property is that a factually wrong draft can still point retrieval at the right neighbourhood. That property is why HyDE is usable as a zero-shot dense method rather than a knowledge source.

Why can a wrong hypothetical still retrieve the right document?

A wrong hypothetical can still retrieve the right document because HyDE uses the draft as a retrieval vector, not as trusted knowledge. Gao et al. (2022) state the generated document “captures relevance patterns but is unreal and may contain false details,” then rely on the encoder’s dense bottleneck to filter incorrect specifics and ground the vector in the neighbourhood of real corpus passages.

The geometry is the same one freeCodeCamp’s HyDE guide (2026) emphasises: embedding models place similar shapes of text near each other. A question and its answer are different shapes; a fake answer and a real answer are the same shape, so they share vocabulary, register and length even when individual facts differ. The metric that matters is whether the retrieved chunks are correct, not whether the scaffolding paragraph was.

With the leak rule fixed, the literature baseline is Gao et al. (2022) Table 1 on TREC DL19 and DL20. Those numbers are for one InstructGPT plus Contriever stack, not a promise for every corpus.

How much does HyDE improve retrieval?

On the web-search sets in Gao et al. (2022, Table 1), HyDE lifts unsupervised Contriever by a large margin and sits close to a Contriever fine-tuned on MS MARCO, without using relevance labels. The published nDCG@10 and recall@1k figures in Gao et al. (2022) Table 1 are taken only from that paper; secondary blog summaries that invent other point gains are not used here.

On DL19, HyDE’s nDCG@10 of 61.3 is within 0.8 of Contriever^FT’s 62.1, and HyDE posts the best recall@1k in that table (88.0). On DL20, HyDE’s nDCG@10 of 57.9 trails Contriever^FT’s 63.2; the paper describes map and nDCG as about 10% lower there, with similar recall@1k. Those numbers are for that InstructGPT plus Contriever stack on those benchmarks as of the 2022 paper.

How to read nDCG and recall in a RAG eval is at retrieval metrics. The next decision is whether the structural cost of a generation call on every query is worth that kind of lift on your own labelled set.

What does HyDE cost at query time?

HyDE’s structural cost is one extra LLM generation call per query before retrieval, plus one embedding of the hypothetical. Baseline dense retrieval makes zero generation calls at retrieve time; HyDE pays that generation cost on every query, not once at ingest. Latency scales with whatever the draft model takes on your stack; measure it rather than trusting a blog’s millisecond claim.

Token spend is paid twice when you also generate the final answer: one call drafts the hypothetical and a later call answers from real chunks. Caching identical queries and using a small, fast model for the draft are the usual mitigations named by freeCodeCamp (2026) and SurePrompts (2026). Cap draft length so over-long hypotheticals do not dilute the embedding; keep the prompt roughly to a paragraph or your median chunk length. If generation times out or errors, fall back to embedding the raw query rather than blocking the request.

Those costs decide when HyDE should run and when a cheaper channel already closes the gap. Identifier-heavy queries and tight p95 budgets are the usual skip cases.

When should you use HyDE?

Use HyDE when the bottleneck is query–document asymmetry and you can afford a generation call on the retrieval path. Short, under-specified questions over prose-heavy corpora such as docs, papers and memos, where users speak lay language and documents speak expert register, are the home case. The method also fits when you lack labelled query–document pairs to fine-tune a dense retriever, which is HyDE’s zero-shot setting in Gao et al. (2022).

Skip HyDE when queries are already long and specific, when they are identifier-heavy (error codes, SKUs, statute numbers), when latency budgets forbid an extra LLM hop, or when the domain is so alien that a general model invents jargon that does not exist in your corpus. Prefer hybrid search and BM25 keyword search for exact-token lookups. Do not confuse HyDE with query rewriting: rewriting cleans or expands a question while it remains a question, while HyDE changes the shape of the search text into a document-register passage. They compose; they are not substitutes. SurePrompts (2026) places HyDE after chunking, hybrid and rewrite in the usual adoption order; measure recall@k on a golden set before making HyDE the default.

Implementation is short once the decision is made: prompt, embed with the same model as the index, search, discard, generate. Runnable wiring belongs with the pipeline build node rather than inside this mechanism.

How do you implement HyDE?

The implementation pattern is short: prompt an LLM for a document-register draft, embed that draft with the same embedding model your index already uses, run ANN search, discard the draft, then generate from the retrieved chunks. Frameworks expose the pattern as LangChain’s HyDE / HypotheticalDocumentEmbedder path and LlamaIndex’s HyDEQueryTransform; verify the current import path as of September 2026 because APIs move.

The vector store that holds the dense index, Weaviate, Qdrant, Milvus, and others, is interchangeable for the search step; HyDE changes what is embedded, not which database you use. Weaviate docs describe hybrid BM25F-plus-vector search as of September 2026 for the lexical backstop case, which is the complementary channel when HyDE is the wrong tool for identifier queries. Runnable, pinned end-to-end code belongs with building the pipeline. Prove whether HyDE earns its latency on Recall@k and nDCG for your own queries; the Gao table is the literature baseline, not your production score.

Teams that adopt HyDE without a golden set often discover the gap only in production: the draft sounds fluent, the nearest neighbours look topical, and the answer still cites the wrong statute or SKU because the search vector never left the wrong neighbourhood. A labelled set that scores recall@k with and without the hypothetical, plus a fail-open path that embeds the raw query on generation timeout, is the minimum operational discipline before HyDE becomes the default retrieve path.

HyDE earns its generation call when short questions sit far from long answers in embedding space and a measured recall@k lift appears on your labels. Keep the hypothetical inside retrieval, discard it before generation, and cite Gao et al. (2022, arXiv:2212.10496) Table 1 as the literature floor rather than any unsourced blog percentage. When the ask is already keyword-shaped or the latency budget forbids an LLM hop, hybrid and BM25 remain the cheaper path; when the ask still needs a cleaner question rather than a fake answer, rewrite first and leave HyDE for the document-register case.