Fixed-Size Chunking: When the Simplest Method Wins
The baseline everything is measured against, its exact failure mode, and the corpora where nothing beats it.
Fixed-size chunking (also called fixed-length or character chunking) splits text into uniform segments of a predetermined length — characters, tokens, or words — with optional overlap between neighbours. It is the deterministic baseline every other strategy is measured against: same input always yields the same chunks, and the splitter never calls an embedding model to decide a boundary. This page covers how the sliding window works, why the unit of measure matters, what the method costs, the corpora where nothing beats it, and the exact failure it creates. The full strategy catalogue lives at chunking.
How does fixed-size chunking decide where to split?
Fixed-size chunking decides boundaries by length alone. The algorithm is a sliding window with four steps:
- Pick a chunk size N. Every chunk targets that length in characters, tokens, or words.
- Pick an overlap O. Neighbouring chunks share the last O units so a short fact that straddles a seam can still land whole in at least one window.
- Extract the window. From the current start position, take the next N units (or to the end of the document).
- Advance by N − O. The next chunk starts before the previous one ends. With no overlap, the step equals N and seams abut with no shared text.
OneUptime’s January 2026 walkthrough uses a worked window on a 1,500-character document at size 500 with overlap 100: the chunks cover ranges 0–500, 400–900, 800–1,300, and 1,200–1,500. The cut points are length arithmetic — not sentence ends, paragraph breaks, or topic shifts. Overlap is the only continuity lever at those seams; how much overlap to use, and how to sweep size, belong on chunk size and overlap.
Should you count characters or tokens when chunking?
Prefer tokens when the budget must match an embedding model or an LLM context window; characters are faster to count but drift from the model’s real unit. Inferensys’s fixed-length glossary is blunt that a “500-token” size is ambiguous until you name the tokenizer — OpenAI’s tiktoken encodings and SentencePiece-style tokenizers split the same string differently. TeachMeIDEA (2026) adds the multilingual trap: a fixed character budget maps to different token counts across languages, so character-sized chunks make embedding cost and prompt size unpredictable. Word counts sit between the two and still ignore tokenizer quirks.
Once the unit is locked, the starting length band on this site is about 500–800 tokens with ~100-token overlap — chosen and measured on chunk size and overlap, not invented here as a universal optimum.
What does fixed-size chunking cost?
Fixed-size chunking is cheap for a structural reason most surveys understate: the splitter makes zero embedding or LLM calls to decide where to cut. It only counts. Semantic chunking pays one embedding call per sentence at ingest to find breakpoints — see semantic chunking. After the split, you still embed each finished chunk once, same as any other strategy.
- Ingest work is I/O-bound. TeachMeIDEA notes that indexing a large corpus with fixed windows is bounded by reading and writing text, not by chunker logic or model inference.
- Capacity planning is arithmetic. Every chunk targets the same length, so approximate vector count from document length and stride (N − O).
- Overlap inflates the index. Extra windows scale with overlap: on a 512-token chunk with 64-token overlap, TeachMeIDEA’s structural ratio is about 12.5% more vectors than zero overlap if you keep stride N − O — arithmetic, not a retrieval benchmark.
Measure dollar and latency deltas on your own corpus
Published blogs rarely publish a controlled $/doc comparison of fixed-size versus recursive on the same stack. The structural fact holds everywhere: zero model calls to place boundaries. Put a real number on wall-clock ingest and embedding spend for your pipeline before you treat “simple” as free.
When does fixed-size chunking win?
Fixed-size chunking wins when documents are homogeneous — consistent structure and density — and when you need a cheap, reproducible baseline before paying for smarter splitters:
- It wins on uniform corpora — simple logs, template pages, short product blurbs, sensor or chat text without meaningful headings (Databricks “Best Fit”; Denser AI “When to Use”; TeachMeIDEA 2026; OneUptime “Good Fit”; Inferensys homogeneous use cases).
- It wins as the baseline cell — any honest chunking sweep should include fixed-size (or a close recursive twin) so you know what “more expensive” strategies buy on labelled queries. Route that experiment to how to evaluate a chunking strategy.
- It barely needs beating on already-flat prose — single-topic text with even paragraphs often chunks similarly whether you cut on length or on separators, so recursive and semantic spend may buy little.
- It loses when structure is free signal — headings, lists, tables, and code want recursive or structure-aware splits; multi-topic long documents lean semantic; gold answers that routinely span seams need overlap tuning then hierarchical retrieval, not a bigger fixed window alone.
As of their 2026 guide, Denser AI cites a Vecta/FloTorch end-to-end comparison reporting 69% accuracy for recursive splitting at 512 tokens versus 58% for fixed-size at the same size on that task — one published comparison, not a universal law. Prove which case you are in by measuring on your corpus, not by copying a blog’s crown.
What failure does fixed-size chunking cause?
Fixed-size chunking’s exact failure mode is an answer cut across a length boundary: the window lands mid-sentence, mid-table, or mid-code block, so no single retrieved chunk holds the whole answer. Inferensys calls this context fragmentation; Denser’s fixed-size demo cuts mid-word on purpose to show the coherence loss. Overlap reduces the odds when the gold span is shorter than the overlap window; it does not remove the failure mode. Fixed-size changes where the cut lands (every N units) — it does not invent a smarter seam. Diagnose SPLIT vs WHOLE on the failure page, tune N and O on chunk size and overlap, and escalate when the split persists.
How do you implement fixed-size chunking?
Every major framework ships a fixed-length splitter. Inferensys’s framework cards name LangChain’s CharacterTextSplitter (character budget + separator), LlamaIndex’s TokenTextSplitter (token budget + tokenizer), and Haystack’s PreProcessor (split_length / split_overlap). The shared knobs are chunk size, overlap, and the length function or tokenizer. Rather than reproduce a full walkthrough here — that belongs with the runnable pipeline — see building the pipeline for a pinned, output-shown version, chunk size and overlap for choosing N and O, and the chunking benchmark for which splitter actually retrieves best on measured data.
What is fixed-size chunking?
Fixed-size chunking (also called fixed-length or character chunking) splits a document into uniform segments of a predetermined length — characters, tokens, or words — with optional overlap between neighbours. Boundaries are decided by counting, not by sentence, paragraph, or topic shifts, so the same input always produces the same chunks.
Should I chunk by characters or by tokens?
Prefer tokens when the budget must match an embedding model or LLM context window. Character counts are faster but drift from the model’s real unit, and the same character budget maps to different token counts across languages and tokenizers. Name the tokenizer when you publish a token size — tiktoken and SentencePiece do not agree.
When is fixed-size chunking good enough?
When documents are structurally uniform — logs, template pages, short blurbs — or when you need a cheap reproducible baseline before paying for recursive or semantic splitting. On already-flat single-topic prose, smarter splitters often buy little. Measure fixed-size against recursive and semantic on your labelled set before you abandon the baseline.
Does overlap fix mid-sentence cuts?
Overlap only helps when the gold answer is shorter than the overlap window, so the full span can land inside at least one chunk. Longer spans stay split no matter how much overlap you add. Tune overlap on /chunking/size; diagnose persistent SPLIT failures on /failures/chunk-boundaries.
What should I measure before leaving fixed-size?
Run the same corpus through fixed-size and at least one structure-aware alternative with the embedding model, retriever, and top-k held fixed. Score recall and precision on labelled questions. The controlled experiment design lives on /chunking/evaluation — the blog crown is not the verdict.