Why RAG Gets Numbers and Tables Wrong
Flattened tables, lost row context and unit errors — the failure path from ingestion to a wrong figure.
RAG gets a table number wrong when the retrievable unit lost the structure that made the cell meaningful — the column header, the row identity, the exact digits, the unit or scale note, or the continuation across pages. Yu et al. (2025, arXiv:2506.10380) state the core limit: flattening tables and naive chunking disrupt tabular structure and undermine reasoning over the cells that held the answer. This page is that failure path. It is not a generic wrong chunk (any irrelevant passage), and it is not the case where the row arrived intact and the model still invented a figure (see hallucination).
Before you change the prompt or the embedding model, read the retrieved chunk text and the extraction artifact that produced it. If the chunk already dropped the header or blended two rows, retrieval never had a clean fact to return — and the model will answer confidently from damaged evidence.
First, which of the five table causes is it?
A wrong table number has five distinct causes, and each has a different fix. Applying a better prompt when the header was never in the chunk changes nothing. Guides that rank for this topic usually stage-walk — fix extraction, then the retrieval unit, then hybrid search, then citations (OptyxStack, 2026) — or ship an extraction cookbook. The faster path is differential diagnosis: start from the one symptom (the figure is wrong), and run the single check that rules each cause in or out before you rewrite the pipeline.
| Cause | What you see | Detection (run this first) | Fix |
|---|---|---|---|
| 1 · Header loss | Right digits; missing or wrong column label | header_preservation | Carry headers in every row chunk |
| 2 · Wrong row | Cites the right document; value from another row | row_hit@k | Structure-aware row chunks |
| 3 · Digit / OCR corruption | Numeric string ≠ source cell | digit_match(source, chunk) | Fix extraction first |
| 4 · Unit / scale error | Magnitude looks right; unit, scale, or footnote wrong | unit_match + scale check | Serialize unit with the value; refuse bare numbers |
| 5 · Multi-page / merged loss | Page-2 rows orphaned; merged label not propagated | continuation_integrity | Merge before chunking |
header_preservation asks whether the gold cell’s column header (and table or section title) appear in the same chunk as the value. row_hit@k asks whether top-k contains the exact answer-bearing row — not merely the right document. digit_match compares the normalized numeric string in the chunk to the source cell. unit_match requires unit or scale tokens from the header or legend in that same chunk. continuation_integrity requires carried-forward headers when the gold row sits on a continuation page or under a merged parent label.
Did the chunk lose the column header?
A retrieved number without its column label is not an answer — it is an orphan value. Header loss is one of the fastest ways table RAG fails in production, because the embedding then represents a weak mini-record instead of a labelled fact (OptyxStack, 2026).
Symptom
The system retrieves “24 hours” but drops the header that said response SLA, not refund window; or retrieves “Allowed” without the adjacent column “Only for internal use” (OptyxStack examples, 2026). Farooq (2026) puts the same point for spreadsheets: the cell 5,000 is meaningless without column “Q3 Projected Revenue” and the row entity.
Detection
For the gold cell, assert that the column header(s) and a table or section title appear in the same retrieved chunk as the value. Measure header_preservation: 1 if present, 0 if absent. Document-level recall can still look fine.
Cause
Flattening or a fixed token window severed the header row from the body, so the retrievable unit no longer names what the number means.
Fix
Repeat the minimum headers inside every row-level unit. How to build those chunks is table chunking; how to keep structure out of the parser is table ingestion.
It cites the right document but the wrong row
Document-level recall can pass while the answer-bearing row never enters the context window. That is why generic retrieval metrics under-detect table failures: they score the page, not the row (OptyxStack, 2026).
Symptom
The answer cites the correct PDF or workbook, but the value belongs to a neighboring row; sibling chunks compete across runs; the model stitches partial rows into a plausible unsupported figure (OptyxStack, 2026). MarkAI Code (2026) describes the same shredding: standard text splitters cut table rows so the LLM gets fragment columns and misaligned headers.
Detection
Measure row_hit@k: did top-k include the exact answer-bearing row id? If document hit is 1 and row hit is 0, stop blaming the embedding model — the unit of retrieval is wrong.
Cause
Rows were blended or split at extraction, or a token-count splitter cut mid-row so no single chunk holds entity + value together.
Fix
Chunk by logical rows with headers carried into each unit — see chunking tabular data. Raising top-k alone is a diagnostic, not a fix. When the query hinges on an ID, plan name, or literal SKU, add a lexical channel via hybrid search.
Did OCR or parsing change the digits?
If the chunk’s numeric string does not match the source cell, retrieval never had a true fact to return — no amount of prompt tuning recovers a digit the parser already corrupted.
Symptom
IDs, SKUs, or percentages are misread; decimal points, currency symbols, or inequality signs disappear; headers and cells are read in the wrong order because layout confused the parser (OptyxStack OCR layer, 2026).
Detection
Run digit_match: normalize the numeric string in the chunk and compare it to the source cell (for Excel, compare to the computed value when the workbook is opened with formulas evaluated). A mismatch means extraction, not ranking.
Cause
OCR noise on scans, or the wrong PDF table mode — bordered “lattice” grids vs whitespace “stream” tables need different parsers (MarkAI Code, 2026). Scanned PDFs need OCR before structure parse; lattice tools that expect a text layer will return empty or garbage tables.
Fix
Repair extraction before embeddings or prompts — OptyxStack’s fix order puts extraction quality first for a reason. The recipes live on handling tables at ingestion.
Is the number right but the unit or scale wrong?
RAG often returns a magnitude that matches a cell while dropping the unit, scale note, or footnote that changes its meaning — “in millions”, “unaudited”, currency, per-share versus total. That is a distinct failure from a wrong digit: the characters look plausible, the quantity is not.
Symptom
The answer says 500 when the table said $500 (millions); mixes a quarterly column with an annual one; or omits a footnote that qualifies the row (OptyxStack footnotes, 2026). KX Systems’ table-heavy RAG guide (ranking on the live 2026-07-27 SERP) groups related mistakes under generation inaccuracy on nested columns — the unit/scale drop is the practitioner form of that class.
Detection
Require unit_match: unit or scale tokens from the header or legend must appear in the same chunk as the number. Flag any answer that emits a bare number when the source cell is labelled.
Cause
Serialization omitted units, or the model renormalized a labelled quantity into a bare figure.
Fix
Serialize header + unit + value as one mini-record at ingest (table ingestion). Instruct abstention on bare numbers. Do not ask the LLM to rescale or total rows in prose — Farooq (2026) notes LLMs are statistical predictors, not calculators; if arithmetic is required, execute it programmatically. When the evidence was intact and the model still invents a scale, that is hallucination, not this cause.
Did a multi-page or merged-cell table lose the fact?
When a table continues across pages or a label spans merged cells, continuation rows often arrive without headers and child rows lose the parent label — humans read the carry-forward; page-wise extractors usually do not (OptyxStack, 2026; Theegala, 2025–2026).
Symptom
Page two repeats partial headers or none; a footnote that changes the row lands far from the cell in flattened text; nested tables or sidebar chrome interleave with the body (OptyxStack hard cases, 2026).
Detection
For a gold row on page > 1 (or under a merged parent), measure continuation_integrity: carried-forward headers and merged-label propagation must be present in the chunk. If they are absent, merge logic failed before retrieval.
Cause
Page-wise extraction without continuation detection — Theegala (2025–2026) treats multi-page tables as the real failure mode and uses horizontal alignment as the strongest continuation signal.
Fix
Detect continuation, merge logically (not only visually), then build row-aware chunks — ingestion owns the merge; table chunking owns the retrievable unit after merge.
How do you run the five table checks on one failing query?
Rather than reason about the five causes one at a time, run them against one labelled failure. You need the failing query, the gold coordinates (document, table, row, column header, value, unit), the retrieved chunk texts, and the source cell string. The function below prints which cause fired. It depends only on string checks and your top-k row ids — no new model required.
Before you trust the numbers
Calibrate on your own table-heavy eval slice. Document-level recall will flatter you; row-hit and header preservation will not. OptyxStack (2026) lists the same eval families — this script binds them as a cause-namer, not a leaderboard score.
import re
def _norm_num(s: str) -> str:
"""Keep digits and one decimal point for digit_match."""
s = s.replace(",", "")
m = re.search(r"-?d+(?:.d+)?", s)
return m.group(0) if m else ""
def diagnose_table_number(chunks, *, gold, top_row_ids, k=5):
"""Print which table-number failure fired for one labelled query.
chunks -> list[str] retrieved chunk texts (top-k order)
gold -> dict with keys:
row_id, col_header, value, unit, page, merged_parent
source_cell (raw string from the PDF/Excel cell)
top_row_ids -> list[str] row ids aligned with chunks (or [] if unknown)
"""
text = "n".join(chunks)
header = gold["col_header"].lower()
unit = (gold.get("unit") or "").lower()
source = gold["source_cell"]
# Cause 1 — header loss: value region without its label
if header and header not in text.lower():
print("CAUSE 1 header loss -> carry column headers into every row chunk")
# Cause 2 — wrong row: document may be present; answer row is not
if top_row_ids and gold["row_id"] not in top_row_ids[:k]:
print(f"CAUSE 2 wrong row (row_hit@{k}=0) -> structure-aware row chunks")
# Cause 3 — digit / OCR corruption
src_n = _norm_num(source)
if src_n and src_n not in text.replace(",", ""):
print("CAUSE 3 digit/OCR mismatch -> fix extraction before embeddings")
# Cause 4 — unit / scale missing beside the number
if unit and unit not in text.lower():
print(f"CAUSE 4 unit/scale missing ({gold['unit']!r}) -> serialize unit with value")
# Cause 5 — continuation / merge loss
needs_cont = int(gold.get("page") or 1) > 1 or bool(gold.get("merged_parent"))
if needs_cont and header and header not in text.lower():
print("CAUSE 5 continuation/merge loss -> merge multi-page tables before chunking")
print("done — delete ruled-out causes; fix the first that remains")Each block is one row of the diagnosis table above. Keep a labelled table slice in your regression set so these checks run in CI — see RAG regression testing. Log table id, row id, column names, header-carried flag, and extraction artifact version when answers depend on tables (OptyxStack telemetry list, 2026); you cannot debug a row you never recorded.
Why are PDF and Excel tables so hard for RAG?
Because the answer usually depends on structure, not just words. Column headers, row identity, units, merged cells, and footnotes give a cell its meaning. If extraction flattens the table into prose or a token splitter cuts headers from values, retrieval may find the right document and still miss the fact.
Is OCR usually the main reason table retrieval fails?
Not by itself. OCR errors matter — wrong digits, dropped decimals, misread IDs — but teams often over-focus on character accuracy while missing lost headers, broken row boundaries, dropped continuation markers, and unit or scale notes that never made it into the chunk.
Should table data be chunked differently from prose?
Yes. A retrievable unit should preserve the row plus the headers and any required units or footnotes. Fixed token windows often cut that unit apart. Structure-aware table chunking is its own page; this page tells you which failure you have before you change the splitter.
What should you measure for table-heavy RAG?
Measure row-hit@k (not only document recall), header preservation in the retrieved unit, digit match to the source cell, unit or scale match, citation validity to the exact row, and continuation integrity on multi-page or merged tables.
The answer cites the right document but the number is still wrong — is that a wrong chunk?
Often it is this failure, not a generic wrong chunk: document-level recall passed and the answer-bearing row, header, digits, or unit did not. Inspect the retrieved table chunk. If an unrelated passage came back, use the wrong-chunk page instead.
Where do the extraction and chunking recipes live?
Diagnosis stays here. PDF and Excel extraction, serialization, and merge-before-chunk patterns live under table ingestion; row-aware and structure-aware splitting live under table chunking.