Chunking by Document Structure: Markdown, HTML and Headings
Using the document own hierarchy as the split boundary, and what to do when the structure is a lie.
Structure-aware chunking splits on the document own hierarchy, Markdown and HTML headings and sections, so each chunk is one authored unit with a heading path as metadata. It respects author boundaries and fails when those boundaries are fake. The strategy catalogue is chunking for RAG; the separator fallback is recursive character chunking.
How does structure-aware chunking decide where to split?
Structure-aware chunking decides splits in three steps: parse typed elements with hierarchy, walk the tree to group content under the same heading, then size-normalize oversized sections without cutting mid fence or mid table. Candidate chunks stay inside authored units rather than arbitrary character windows.
Parse into headings, paragraphs, lists and fences with levels. Walk the tree so content under one heading becomes a candidate. Never split mid code fence or mid table; those formats are owned by code chunking and table chunking.
Size-normalize next: too big means split by sub-heading then recursive within the section; too small means merge only within the same section. Recursive character splitting only knows separators, not a heading tree, which Medium-style write-ups treat as similar intent but not the same method.
Markdown header splitters are the cleanest production path when sources are already Markdown. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
How does Markdown header chunking work?
Markdown header chunking splits on a declared header list so each Document carries the header path in metadata, then oversized sections are sub-split with a recursive character splitter inside the section. LangChain MarkdownHeaderTextSplitter in langchain-text-splitters is the common reference pattern as of mid-2026 docs.
Configure headers_to_split_on such as Header 1 for hash-one, Header 2 for hash-two, Header 3 for hash-three. Output Documents carry Header 1, Header 2 and Header 3 keys; by default headers are stripped from page_content unless strip_headers is false.
Production pattern from LangChain docs and practitioner write-ups: header-split first, then RecursiveCharacterTextSplitter.split_documents on oversized sections so overlap stays inside a section and does not cross H1 or H2 boundaries. LlamaIndex MarkdownNodeParser follows the same idea.
HTML sources need DOM-aware splitters or a clean conversion to Markdown before the same header path applies. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
How does HTML chunking work for RAG?
HTML chunking parses the DOM, identifies content containers, and splits on heading tags or section boundaries so retrieved units still carry authored structure instead of tag soup. LangChain ships HTMLHeaderTextSplitter, HTMLSectionSplitter and HTMLSemanticPreservingSplitter in the text-splitters package; LlamaIndex offers HTMLNodeParser.
IBM cookbook guidance breaks on tags such as p and div, extracts main text and headings, and warns that older HTML with non-standard tags, nested deprecated markup and inline style clutter produces fragmented chunks. Practical pattern: convert clean HTML to Markdown then reuse the Markdown header splitter, or split headers in place.
Azure Document Layout skill detects headings from layout in PDF, DOCX and HTML into Markdown-like header fields, then a text split constrains size, as documented on Microsoft Learn. Parser quality dominates; layout detection cannot invent true topics from decorative chrome.
Heading path metadata is the next rule once splits exist. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
Why should each chunk carry the heading path?
Every structure-aware chunk should carry its heading path in metadata or prepended text so embeddings and the generator know which section the passage belongs to. Without the path, repeated titles such as Overview or Limitations collide across documents and sections.
Ochoa-style heading context trick: Setup then Authentication then OAuth then Configuration disambiguates rate limits across API, OAuth and webhook sections. unrag-style guidance prepends parent headings so subsection chunks are self-describing. dataknobs-style pipelines store the full path from root to chunk in metadata. LangChain stores Header 1, Header 2 and Header 3 keys on Document.metadata.
Heading path is required here rather than a separate metadata-enriched strategy. Parent retrieval that expands small children into larger parents is owned by hierarchical chunking once real parents exist. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
When headings lie, path metadata amplifies noise instead of fixing it. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
What do you do when the document structure is a lie?
When document structure is a lie, structure-aware chunking amplifies noise, so validate heading density and chrome before trusting the tree, then fall back to recursive or fixed splits or fix parsing upstream. Decorative or SEO headings that do not mark topics are the first lie mode.
Nav, chrome and sidebar headings parsed as H2s, especially on messy scrapes and older HTML, create fake sections. PDF to Markdown or OCR that invents levels from font size depends entirely on parser quality. Stub headings with one sentence yield weak embeddings. Missing headings yield one giant chunk or useless fragments.
Fallback rule: validate structure for heading density, empty stubs and chrome filters. If unreliable, use recursive character chunking or fixed-size chunking, or fix parsing and cleaning upstream at ingestion cleaning. If reliable, keep structure plus recursive sub-split. Do not invent a percentage of documents that lie.
Knowing when the method wins versus recursive is the next decision. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
When should you use structure-aware chunking instead of recursive?
Use structure-aware chunking when technical docs, wikis, manuals or Markdown and HTML sources carry a real heading hierarchy, and prefer recursive when separators exist but no reliable tree is present. Plain prose blogs without H2s, chat logs, emails and homogeneous short posts barely benefit and can be overkill.
Structural cost at split time is zero embedding or LLM calls: string or DOM parse only, the same cheap class as recursive, unlike semantic chunking which embeds adjacent units. After structure parents exist, escalate long-answer needs to hierarchical chunking.
Prove the choice on chunking evaluation with labelled questions. Unpublished Top-1 percentage tables from blog bake-offs are not cited here because methods are not independently reproducible. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
Implementation is a class-and-pattern handoff rather than a full notebook. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
How do you implement structure-aware chunking?
Implement structure-aware chunking with MarkdownHeaderTextSplitter then RecursiveCharacterTextSplitter.split_documents for Markdown, or HTMLHeaderTextSplitter for HTML, pinning langchain-text-splitters and verifying imports before shipping. LlamaIndex MarkdownNodeParser and HTMLNodeParser cover the same idea; Azure Document Layout plus text split covers layout-detected headers.
Runnable pinned pipelines live at building a RAG pipeline. Size and overlap knobs after header split live at chunk size and overlap. Measure lifts at chunking evaluation. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
Store heading path properties on Weaviate objects, and keep the same metadata keys if Pinecone or Qdrant peers are dual-run during migrations. Reject chrome-heavy parses before upsert. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
Structure-aware chunking is hierarchy-first splitting with an explicit lie fallback, not a guarantee that every HTML scrape is trustworthy. Operators should treat this as a hard requirement when comparing chunkers on the same labelled set.
Operational control 1: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 2: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 3: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 4: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 5: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 6: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 7: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 8: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 9: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 10: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 11: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 12: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 13: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 14: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 15: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 16: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 17: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 18: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 19: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 20: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 21: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 22: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 23: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 24: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 25: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 26: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 27: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 28: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 29: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 30: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Operational control 31: freeze splitter version and header rules beside the Weaviate collection identifier, dual-run Pinecone or Qdrant peers when present, and promote only after labelled metrics clear a pre-registered threshold.
Structure-aware chunking splits on authored headings so chunks stay section-shaped and carry heading paths. Markdown and HTML splitters plus recursive sub-split are the production pattern. When headings are decorative, OCR-invented or chrome, fall back to recursive or fixed paths and fix parsing upstream. Measure on labelled questions before fleet promotion.