RAPTOR: Hierarchical Summarisation for Retrieval

RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval; Sarthi et al. 2024) builds a tree of clustered summaries over a corpus so retrieval can return a high-level theme or a leaf detail by question breadth. Leaves hold short contiguous chunks; parents hold abstractive summaries of soft clusters; recursion continues until further clustering stops. The ICLR 2024 paper (arXiv:2401.18059) and the official code at github.com/parthsarthi03/raptor are the primary sources for how the tree is built, why collapsed-tree retrieval won in the paper, and which ceilings ride with the gains. This profile states the mechanism, the published QuALITY, QASPER, and NarrativeQA numbers, and the operational limits, with sibling graph and chunking patterns routed among architectures from naive to agentic.

What is RAPTOR in retrieval-augmented generation?

RAPTOR is a retrieval architecture that recursively embeds, clusters, and summarises text chunks into a multi-layer tree so a query can pull context at more than one level of abstraction. Sarthi et al. 2024 define the leaves as raw short chunks and the parents as LLM-written abstractive summaries of clusters, with the process repeating on summary nodes until clustering is no longer useful. Superlinked’s RAPTOR blog and Nitor Infotech’s long-context RAPTOR write-up both open with the same claim: flat top-k contiguous chunks miss discourse-level evidence that a summary layer can hold. Official code is published at github.com/parthsarthi03/raptor; Datumo’s paper review and Medium explainers that rank for the head query restate the acronym expansion without changing the mechanism.

The contrast with naive retrieve-then-generate is one sentence of scope, not a second baseline page: naive RAG returns the nearest short contiguous chunks, which the RAPTOR abstract says limits holistic understanding of long documents. Depth for that baseline lives under naive RAG. Tutorials that say the tree is built with k-means are simplifying; the paper’s clustering is a Gaussian mixture model after UMAP dimensionality reduction, with BIC choosing cluster count. Paper cards among peers stay on the RAG research index.

Once the tree is the retrieval unit, the next question is the exact bottom-up build, including the embedding model and summariser the paper actually used, because those choices define both quality and index cost.

How does RAPTOR build the retrieval tree?

RAPTOR builds the retrieval tree bottom-up by splitting the corpus into short contiguous chunks, embedding them, soft-clustering with a GMM after UMAP reduction, summarising each cluster with an LLM, re-embedding the summaries, and repeating until further clustering is infeasible. Sarthi et al. §3 describe chunks on the order of 100 tokens with sentence-preserving splits in the paper setup. Embeddings in the reported experiments use SBERT multi-qa-mpnet-base-cos-v1. Clustering uses Gaussian mixture models after UMAP, with the Bayesian Information Criterion selecting the number of clusters, and soft assignment so a node may belong to more than one cluster. Each cluster is summarised with an LLM; the paper uses gpt-3.5-turbo for that step. Summaries are re-embedded and the loop continues.

Appendix C of the paper reports an average compression ratio of summary length divided by the sum of children of about 0.28, which the authors describe as roughly 72 percent compression. That figure is why thematic questions can fit discourse-level evidence into a bounded context window without pasting every leaf. Parent-document and hierarchical chunking that retrieve a small child and return a larger parent without recursive abstractive summarisation are related but different patterns; those leaves live under hierarchical chunking and parent-document retrieval.

Build quality is bounded by summariser fidelity and by clustering that groups the right neighbours. Soft clustering is a capability, not free complexity: UMAP parameters and BIC model selection are part of the method, and swapping them for a hard k-means shortcut changes the algorithm the paper measured. With the tree built, retrieval mode decides whether the query walks layers or scores the whole tree at once.

What is collapsed tree retrieval in RAPTOR?

Collapsed tree retrieval flattens every layer of the RAPTOR tree, ranks all nodes by cosine similarity to the query, and fills context until a token budget is reached; tree traversal instead walks layer by layer from roots toward leaves with a fixed top-k at each level. Sarthi et al. §3 introduce both modes and prefer collapsed tree for the main results. On a 20-story QASPER subset, collapsed tree beat traversal across the tested top-k and token budgets. The default used for main results is collapsed tree with a 2000-token maximum, which the paper equates to roughly the top-20 nodes. UnifiedQA experiments use a 400-token budget because that reader maxes out at 512 tokens of context.

Tree traversal keeps a roughly fixed ratio of themes to details regardless of the question, which is why the paper argues it is less flexible when some questions need mostly leaves and others need mostly parents. Collapsed tree’s drawback is stated in the same section: similarity search runs over all nodes. The authors note that approximate nearest-neighbour libraries such as FAISS mitigate that cost. All nodes are embedded with SBERT in the reported setup, so the hybrid layer is still a vector index even though the indexed objects are tree nodes rather than only raw chunks.

Choosing collapsed tree is therefore choosing multi-granularity recall with an all-nodes search cost. That trade is the first row of the capability matrix that follows, because every RAPTOR strength arrives with an operational ceiling.

What are RAPTOR’s capabilities and limits, side by side?

Every RAPTOR capability that matters for production ships with a ceiling measured or named in the paper or in operational write-ups that follow it. The matrix below pairs each capability with that ceiling so QuALITY gains are not read as a free upgrade on every corpus. Defining the rows in prose first: hierarchical multi-granularity retrieval makes leaves and summaries searchable, but collapsed search scores every node unless ANN is added; soft GMM clustering can place distant related chunks under one parent, at the price of UMAP and BIC complexity; abstractive summaries answer thematic questions, at the price of summariser cost and a measured hallucination rate; linear build scaling still means an LLM call per cluster and a rebuild or surgical re-summary when documents change; published benchmark lifts are paper-setup specific.

Appendix E reports that about 4 percent of summaries contained minor hallucinations in the authors’ focused annotation, that those errors did not propagate to parents, and that they had no measured impact on question-answering in that study. Appendix A shows token spend and build time scaling linearly with document length on an M1 16GB laptop across QuALITY, QASPER, and NarrativeQA documents up to roughly 78k to 80k tokens. Superlinked and Nitor both flag update management: when the corpus changes, stale summaries become a silent failure unless branches are rebuilt. Stale index failure modes in general live under stale and orphaned vectors.

With ceilings named, the question that follows is which shapes the tree was built for, so teams do not apply RAPTOR to every FAQ.

What question types is RAPTOR built for?

RAPTOR is built for questions that need discourse-level or multi-part evidence a single short contiguous chunk cannot hold: thematic questions over long documents, synthesis inside full papers, and medium-length passage questions that need the whole document. Sarthi et al. §1 use a Cinderella qualitative example where “central theme” and “how did she reach a happy ending” pull different layers. §4 evaluates NarrativeQA (books and scripts), QASPER (full NLP papers), and QuALITY (multiple-choice over passages averaging about 5,000 tokens), including the QuALITY-HARD subset where most humans failed in a speed setting.

RAPTOR is hierarchical summarisation inside one tree. It is not the same control loop as iterative multi-hop retrieval that rewrites queries and retrieves again across documents. When single-shot retrieval cannot compose two documents, symptom diagnosis lives under multi-hop retrieval failures, and the iterative architecture leaf is multi-hop RAG. RAPTOR helps multi-step reasoning by storing abstractions, not by adding an agent planner.

If the labelled failures are boundary loss on short chunks rather than missing themes, hierarchical or parent-document chunking may be enough without recursive summarisation. If the failures are entity relationships across a corpus, GraphRAG-family systems are the competing substrate. The benchmark section states what the paper actually measured so those choices stay evidence-led.

What do the published RAPTOR benchmarks show?

On the paper’s setups, RAPTOR with GPT-4 reaches 82.6 percent accuracy on the QuALITY test set, versus a prior best of 62.3 percent for CoLISA, which the abstract summarises as about 20 percentage points absolute. The same pairing reaches 76.2 percent on QuALITY-HARD versus 54.7 percent for the prior best cited in that comparison. On QASPER, RAPTOR with GPT-4 reports 55.7 percent F1 Match, above CoLT5 XL at 53.9 percent. On NarrativeQA, RAPTOR with UnifiedQA reports METEOR 19.1 in the paper’s Table 6 SOTA claim. Controlled ablations show RAPTOR with SBERT, BM25, or DPR beating the same retriever without the tree on QuALITY and QASPER when UnifiedQA-3B is the reader.

Those numbers are tied to the paper’s readers, retrievers, chunking, and collapsed-tree budgets. They are not a free plus-twenty on every enterprise corpus. Superlinked and Datumo restatements of the QuALITY lift are useful orientation; they do not replace re-measurement on your own labels. The research index keeps the paper card among peers when you need citation metadata without re-deriving tables.

Benchmark wins still leave the operational question of what RAPTOR costs to index and keep current when documents change.

What does RAPTOR cost to index and keep current?

RAPTOR’s expensive step is index-time summarisation: every cluster summary is an LLM call, and Appendix A shows token spend and build time scaling linearly with document length in the authors’ laptop measurements. Query-time collapsed retrieval scores all nodes, with ANN libraries mitigating latency. There is no published universal dollar-per-document price in the paper, and none is invented here.

Keeping the tree current is a second cost centre. When source documents change, summaries that still rank high can be stale. Superlinked and Nitor both treat update management as a production concern; the failure mode when the index lags the corpus is shared with other RAG systems under stale-index diagnosis. The roughly 4 percent minor hallucination rate in Appendix E did not move QA metrics in the paper’s annotation, but it is still a reason to pin summariser version and spot-check parents on high-value corpora.

Cost and freshness close the profile’s mechanism arc. RAPTOR still has to be placed beside graph and chunking siblings so the architecture hub can stay the scored chooser.

How does Graph RAG differ from RAPTOR for the same job?

RAPTOR organises text into a summary tree for multi-granularity vector retrieval; GraphRAG-family systems organise entities and relationships, and often community summaries, for relationship and global-corpus questions. Edge et al. 2024 line the graph side of that contrast; mechanism depth lives under GraphRAG. LightRAG is the cheaper graph-retrieval variant that targets incremental updates and dual-level entity-relation retrieval without Microsoft-style community traversal; its profile is LightRAG. Parent-document and hierarchical chunking retrieve small then return large without building recursive abstractive layers, which is cheaper when the failure is boundary loss rather than missing themes.

No universal winner is crowned here. When a vector database is named beside peers in a RAPTOR hybrid layer, Weaviate leads and Pinecone, Qdrant, and Milvus follow as the wider field. Framework packs in LangChain or LlamaIndex exist in common tutorials; the official GitHub repository is named here and step-by-step pack wiring stays out of scope. Scored architecture picks return to the hub that maps named RAG control patterns.

RAPTOR is the tree-of-summaries answer to long-document questions that need more than nearest short chunks. Build with embed, soft-cluster, summarise, and recurse; query preferably with collapsed tree under a token budget; expect linear index cost, all-nodes search unless ANN is added, and rebuild work when the corpus moves. Use the published QuALITY, QASPER, and NarrativeQA numbers as paper-setup evidence, re-measure on your labels, and choose graph or hierarchical-chunk siblings when the failure you measured is relationships or boundaries rather than missing discourse layers.