Chunking for RAG: How to Split Documents That Retrieve Well

Chunking is how you split documents into the passages retrieval can return — and those cut points decide what the model ever sees. Pick the strategy that matches your corpus, then measure it on labelled queries. This page is the map; depth lives on the children.

What is chunking in RAG?

Chunking in RAG is breaking source documents into smaller passages — chunks — that are embedded and indexed as the smallest units a retriever can return. Multimodal, Redis, and Microsoft’s Azure RAG guidance (all live on the SERP for rag chunking, captured 2026-07-27) place it at index time: after documents are loaded and cleaned, before each passage is embedded into a vector store. The cut points define what can ever come back at query time — a fact split across two chunks, or buried inside a multi-topic chunk, cannot be recovered cleanly by the embedding model alone.

Strategies sit on a spectrum from cheap length cuts to expensive context-aware splits. The diagram is the whole field in one view — every box routes to a page that owns the mechanism.

Why is chunking important in RAG?

Chunking is important because embedding models have fixed maximum sequence lengths — text past the limit is truncated — so documents must be split while preserving enough meaning to retrieve and answer (Redis, Chunking for RAG, live 2026-07-27). The second structural fact is cost: Microsoft’s Azure RAG guidance frames real chunking economics — engineering cost to design each approach, plus per-document processing cost that differs by method (Microsoft Learn, RAG chunking phase, live 2026-07-27). Semantic and LLM-driven splitters sit at the expensive end of that spectrum; fixed and recursive sit at the cheap end.

When the cut is wrong, the failure shows up as retrieval, not as a “bad model”:

• Answer split across a boundary — no single chunk holds the whole fact. That is chunk-boundary loss.
• Diluted multi-topic chunk — the right passage is buried inside noise, so ranking returns the wrong unit. That is the dilution side of wrong-chunk retrieval.

There is no universal “best” strategy on this page — ranking guides that crown one default still disagree on size, overlap, and when semantic is worth the ingest tax. Prove the choice with a controlled chunking experiment; the length and overlap knobs that every splitter still carries live on chunk size and overlap.

Which chunking strategy should you use?

Match the decision you are actually making to a path below — then open the leaf that owns the mechanism, the size knobs, or the measurement. Depth lives on those pages; this hub only orients. A defensible starting path on the top-ranking results (Digital Applied’s decision playbook; Atlan’s strategy framework; Azure’s match-approach-to-document-structure — all 2026-07-27) is: begin with recursive or structure-aware splits on structured corpora, and graduate to semantic, hierarchical, late, or agentic only when your labelled metrics justify the added cost.

Length and separators — the baseline

Meaning and hierarchy — split on ideas

Document shape — honour the format

Context enrichment — keep more than the window

Prove it — measure before you commit