Recursive Retrieval

Recursive retrieval first hits compact references — summaries, small chunks, or index nodes — then follows those references down to the fuller text nodes that actually answer, instead of returning only the first similarity hits. DataCamp’s LlamaIndex tutorial describes the pattern as retrieving relevant summaries first, then drilling into the linked chunks; LlamaIndex’s Recursive Retriever docs call the same idea traversing node relationships to fetch nodes based on references. This is not recursive character chunking (an ingest-time separator hierarchy) and not RAPTOR (a recursively built summary tree at ingest).

How does recursive retrieval follow references?

Recursive retrieval follows references in a graph of nodes: the first hit is often a pointer, and the retriever expands that pointer into the text the generator should read.

1. Index leaves and reference nodes together. Leaf nodes hold the passages (or tables / sub-documents) you may eventually return. Reference nodes — summaries, smaller chunks, or LlamaIndex IndexNode objects — store a pointer to those leaves (LlamaIndex Recursive Retriever + Node References guide).
2. First retrieval returns references. Similarity search can surface a summary or child chunk whose embedding matches the query better than the long leaf text would.
3. Traverse and fetch. A recursive retriever walks the reference edges and loads the pointed-to nodes — Pondhouse Data (2024) summarises the two required pieces as (a) relationships between chunks and (b) a way to retrieve related chunks recursively.
4. Stop on a depth or score budget. Without a hop cap, follows can fan out through every cited page or linked summary in the graph.

What kinds of node references does recursive retrieval use?

Recursive retrieval uses different reference shapes for the same follow-the-pointer walk. LlamaIndex’s docs group two engineering patterns; Chia Jeng Yang (Knowledge Graph RAG / Medium, Jan 2024) groups three intent types.

LlamaIndex reference patterns:

• Chunk references. Smaller child chunks refer to a bigger parent chunk so search stays precise while generation can expand (related to, but not identical to, parent-document retrieval).
• Metadata references. Summaries and generated questions refer to a bigger chunk — the query matches the metadata node, then recursion loads the source text (LlamaIndex Recursive Retriever + Node References).

Yang’s three recursive-retrieval types (2024):

• Page-based. Follow explicit page or exhibit citations in manuals the way a reader flips to “see page 10.”
• Information-centric. Fix a seed concept and keep retrieving information that attaches to that node.
• Concept-centric. Let the model propose next concepts (n+1 nodes) and walk the graph until the answer is grounded — harder to stop cleanly than a look-up.

What does recursive retrieval cost?

Recursive retrieval’s cost is structural: every follow-up hop is another retrieval (and sometimes another sub-query engine call), and building summary or question reference nodes at ingest adds LLM calls before any user query.

• Query-time hops. Each recursive expand hits the index or a linked retriever again, so latency and token assembly scale with depth and fan-out.
• Ingest-time reference construction. Summary-index and question-metadata patterns (as in DataCamp’s LlamaIndex walkthrough) pay generation cost once per document or chunk when those reference nodes are created.
• Depth must be capped. Yang notes that concept-centric search can keep discovering nodes without a clear “done” signal — treat max hops / max nodes as a product control, not an optional debug flag.

When should you use recursive retrieval?

Recursive retrieval earns its hops when the answer sits behind a reference the first hit only points to — not when a flat top-k already returns a complete passage.

• Use it for citation-heavy manuals. Page-based recursion matches corpora that say “see section 4.2” or “refer to page 20” (Yang, 2024).
• Use it when summaries retrieve better than raw chunks. Large collections where chunk embeddings are noisy benefit from summary-first then drill-down (DataCamp).
• Use it for tables linked to surrounding text. Pondhouse and LlamaIndex’s PDF-tables recursive-retriever demo attach table nodes (and query engines) behind references so the table is not orphaned from its caption text.
• Prefer a simpler sibling when one expand is enough. If you only need child→parent expansion once, start with parent-document retrieval or auto-merging instead of a multi-hop reference walk.

How is recursive retrieval different from recursive chunking or RAPTOR?

The word “recursive” names different mechanisms on this site. Recursive retrieval is the query-time reference walk; the siblings below are not drop-in synonyms.

• Recursive character chunking splits text at ingest with a separator hierarchy (paragraph → line → space → character). File: /chunking/recursive (chunking-recursive.html). This page’s file is retrieval-recursive.html for the same last URL segment.
• RAPTOR recursively embeds, clusters, and summarises chunks into a multi-layer tree at ingest, then retrieves from those layers (Sarthi et al., 2024). Profile: RAPTOR.
• Parent-document retrieval expands child hits to parents once; it does not walk an arbitrary reference graph hop-by-hop.
• Multi-hop RAG rewrites or decomposes the query across documents; recursive retrieval follows document references. Loop depth: multi-hop RAG.

How do you implement recursive retrieval?

Implementing recursive retrieval is a handoff checklist: build the reference graph, attach retrievers, wrap a recursive retriever with caps, and benchmark against flat top-k.

1. Build reference nodes. Create summary, question, child-chunk, or table IndexNode objects that point at leaf text (LlamaIndex pattern).
2. Attach per-subgraph retrievers. Top-level summary index plus document-level or table-level retrievers, as in DataCamp’s and Pondhouse’s LlamaIndex guides.
3. Wrap with a recursive retriever. Set similarity top-k and a maximum recursion depth so follows cannot unbounded-walk the graph.
4. Evaluate against a flat baseline. Same queries, with and without recursion — especially on manuals and table-heavy PDFs.

Runnable wiring belongs on building the pipeline.