Chunking Source Code for RAG

Code chunking for RAG splits source at syntactic units — functions, classes, methods — not every N characters. Character and fixed-size cuts bisect functions and drop brace closure; AST-based chunking parses the file and cuts at tree boundaries so retrieved units stay whole. This page covers why prose splitters fail on code, how Abstract Syntax Tree (AST) splitting works, how LangChain’s from_language helpers differ from a true parse, and when to reach for each. The strategy catalogue lives at chunking; repo-level product patterns live at RAG for codebases.

Why does character splitting destroy code retrieval?

Character and fixed-size splitters treat source as prose. On code that produces fragments that neither compile in the reader’s head nor embed as a complete unit:

• Mid-function cuts. A token or character window can end inside a call — for example after const result = await db.query( — and start the next chunk on the SQL string and closing braces (the failure walk-through on unrag.dev’s code-chunking docs). The first chunk has no return path; the second has no function name.
• Unclosed syntax. VXRL’s March 2025 write-up runs RecursiveCharacterTextSplitter at chunk_size=50 on a short C++ file and gets chunks that open void greet without its body, or close braces without the signature that owns them.
• Sentence-level cuts on code are worse. The “25 chunking tricks” notes that ranked for this query (DEV / Medium, 2026 SERP) report sentence splitting a Python script into tokens like def, return, and if as separate “answers” — worse than fixed windows.

The embedding model faithfully encodes the fragment. A query such as “how to create a user” then misses or ranks half-functions. Fixed-size chunking is the clean baseline that exposes this; recursive character chunking with prose separators still uses character logic and will cut the same way unless you swap in language-aware separators or an AST path.

# Chunk A — incomplete
export async function getUser(id: string): Promise<User | null> {
  const result = await db.query(

# Chunk B — orphan body
    "SELECT * FROM users WHERE id = $1", [id]);
  return result.rows[0] ?? null;
}

AST-aware splitting keeps getUser in one chunk (and createUser in another) so a create-user query can retrieve a complete function — the same contrast unrag.dev draws on a TypeScript sample.

How does AST-based code chunking work?

AST-based code chunking is a parse-then-pack loop, not a character counter:

1. Parse the file into an Abstract Syntax Tree. A grammar (commonly tree-sitter in production libraries) turns source into typed nodes — function definitions, classes, imports — with byte ranges.
2. Keep whole nodes when they fit. Walk top-down and place a complete syntactic unit in one chunk whenever it sits under the size budget.
3. Split oversized nodes; merge small siblings. Zhang, Zhao, Wang, Yang, Wei, and Wu’s cAST method (arXiv:2506.15655; EMNLP 2025 Findings) recursively breaks large AST nodes into children, then greedily merges adjacent siblings that still fit — “recursive split-then-merge” — so you avoid both mid-function cuts and a pile of one-line scraps.
4. Measure size in non-whitespace characters. Two spans with the same line count can carry very different amounts of code; cAST (and production ports such as Supermemory’s code-chunk write-up) budgets non-whitespace characters so blank lines and comment padding do not fake a full chunk.

Imports and tiny declarations often group with a neighbour; a single function larger than the budget still splits, preferably at statement boundaries rather than mid-expression (unrag.dev). At split time the cost is CPU parse plus native grammar bindings — zero embedding calls to place cuts, unlike semantic chunking, which embeds every sentence to find breakpoints. After boundaries exist you still pay one embedding per final chunk, as with any other splitter.

On the authors’ published setups, cAST reports Recall@5 up by 4.3 points on RepoEval retrieval and Pass@1 up by 2.67 points on SWE-bench generation versus their line/fixed baselines (Zhang et al., abstract, 2025). Those deltas are measurements on those benchmarks and retrievers — not a universal lift for every repo. Prove the gap on your own chunking evaluation.

Is LangChain’s code splitter the same as AST chunking?

No. LangChain’s documented code path (Python integrations “Splitting code” docs, captured July 2026) is RecursiveCharacterTextSplitter.from_language with a Language enum — a prebuilt list of string separators per language (class/def-style boundaries and similar). LanceDB’s 2024 chunking survey describes the same “Code Splitter” pattern. That is still the recursive character algorithm from recursive chunking; it does not build an Abstract Syntax Tree.

True AST chunking (cAST / the astchunk companion package, or tree-sitter libraries such as Supermemory’s code-chunk) parses typed nodes and packs them under a size budget. from_language is a real upgrade over prose separators and is often enough on tidy, conventional files; it can still miss atypical syntax the separator list does not name. Rule of thumb: language-aware separators = better recursive; AST = structure-aware parse.

When should you use AST code chunking?

AST (or at least language-aware) code chunking earns its complexity when the retrieval unit is a syntactic definition:

• Use it for codebase search and coding assistants — indexing .py / .ts / .go and similar so “how does auth work?” can return a whole function or method (unrag.dev’s when-to-use list; denser’s 2026 guide routes file_type == code to code-aware splitting).
• Prefer something else for prose and Markdown docs — recursive or semantic for natural language; document-structure chunking when fenced code blocks sit under headings and must stay intact with their section (unrag.dev explicitly routes Markdown with code fences away from the code chunker).
• Plan for parse failure. Invalid or unsupported syntax should fall back to line/text splitting and surface a warning (unrag.dev’s code_parse_fallback) — do not pretend an AST run succeeded. Tiny single-function files already under chunk_size gain little from a full parse.

Treat from_language as a measured intermediate before committing to native AST dependencies. Compare strategies on labelled retrieval metrics, then frame the product layer — repo ingestion, agent tools, internal docs — on RAG for codebases.

What failure does code chunking prevent?

Code chunking exists mainly to stop one failure: a function or method split across two chunks because a character window cut mid-body. That often looks like a retrieval or generation bug — half an answer, missing parameters, uncompilable completions — and is frequently a chunking bug. Structure-aware cuts reduce mid-function SPLITs; they do not remove cross-file multi-hop needs, and they do not stop retrieving the wrong whole unit. Oversized methods still must split somewhere, preferably at statement boundaries, so boundary loss remains possible inside giant functions. Chunking changes the odds; it does not delete the failure mode.

How do you implement code chunking for RAG?

Two practical paths, as of the July 2026 docs and library pages:

• Language-aware recursive — RecursiveCharacterTextSplitter.from_language in the langchain-text-splitters package (LangChain’s code splitter docs).
• AST / tree-sitter — Zhang et al.’s companion astchunk on PyPI, or other tree-sitter-based chunkers (for example Supermemory’s code-chunk; unrag’s chunker:code). Native grammars usually need a working C/C++ toolchain at install time.

Pin versions; do not ship unpinned “latest”. A runnable end-to-end pipeline belongs at building the pipeline; size and overlap knobs at chunk size and overlap; which splitter wins on measured data at chunking evaluation; codebase product patterns at RAG for codebases.