Context Assembly: Building the Prompt the Model Actually Sees
Retrieval returns candidates; assembly decides what reaches the model. The stage almost no tutorial names.
Context assembly in RAG is the stage after retrieval that decides which retrieved chunks enter the prompt, in what order, under what token budget. Find the packing problem, open the leaf.
What is context assembly in RAG?
Context assembly in RAG is the post-retrieval stage that turns a ranked candidate list into the prompt context the model actually sees — selecting, budgeting, ordering, compressing and deduplicating chunks before generation. Unrag.dev’s packing guide puts it as arranging retrieved content (ordering, formatting, labeling) so the model can extract and synthesize; Praxen’s context-packing notes (Medium) call the same object a budgeted evidence package. Retrieval returns candidates; assembly decides what reaches the model; generation answers from that package alone.
That is a different job from Anthropic’s contextual retrieval, which prepends document context to each chunk at index time before embedding. Index-time context helps retrieval score better; assembly packs whatever retrieval already returned into the prompt the generator reads.
The pipeline beat, in order:
- Candidates in — the top-k (or reranked) passages from retrieval.
- Budget — reserve tokens for instructions, history and generation; retrieved evidence gets the remainder.
- Dedupe — drop near-duplicates that waste the budget and skew the answer.
- Order — place the strongest evidence where the model attends.
- Compress — shrink what still overflows without discarding the claim you need.
- Prompt out — hand the packed context to the model.
How does context packing work after retrieval?
Context packing after retrieval works by spending a fixed token budget across instructions, history and retrieved evidence — then dropping near-duplicates, placing the best chunks where the model attends, and compressing what still overflows. Unrag.dev’s token-budget section is explicit: context cannot consume the entire window; reserve essentials first, then fill the remainder with evidence. The four packing decisions map to this hub’s children:
- Window budget — how many tokens retrieved content may use versus system instructions and chat history, and what overflows first. Depth on budgeting the context window.
- Dedup — near-duplicate chunks waste tokens and skew the answer toward whatever was repeated. Unrag.dev and Praxen both put semantic dedupe before packing. Depth on removing redundant chunks.
- Order — position changes whether a retrieved fact is used. Liu et al. (2023; TACL 2024) measured a U-shaped accuracy curve on multi-document question answering: models under-use mid-context evidence. Redis’s February 2026 RAG-vs-window write-up summarises the same literature as 10–20+ percentage-point drops when gold sits in the middle (verify the per-model table on the failure leaf before you rely on a single delta). Reorder on where to put the best chunk; diagnose the symptom on lost in the middle.
- Compress — extractive or abstractive shrink keeps the claim you need under budget. Depth on contextual compression.
Token cost still scales with what you pack. Redis (February 2026), citing OpenAI’s then-listed GPT-4.1 prices, quotes $2.00 per million input tokens and $8.00 per million output tokens — so a 100,000-token input alone is about $0.20 before output (verify current list prices before you rely on them). A bigger model window does not remove the packing job; the scored RAG vs long-context comparison covers when stuffing the corpus still loses on cost, latency or mid-context under-use. How those chunks are wrapped (XML tags, numbered sources) and the grounding instructions around them are prompt design on writing the RAG prompt — not a fifth packing lever on this hub.
What context problem are you solving?
Match what you are seeing to a packing problem below — described in the words you’d use — and follow it to the leaf that owns the mechanism.
Fit the window — tokens are the constraint
Shape the prompt — what the model attends to
Already retrieved, still wrong?
If the gold chunk was in the prompt and the model still missed it, that is usually position bias — start at lost in the middle. If you are deciding whether a million-token window replaces retrieval entirely, that verdict is RAG vs long context.
What is context assembly in RAG?
Context assembly is the post-retrieval stage that turns ranked candidate chunks into the prompt context the model actually sees — by budgeting tokens, dropping near-duplicates, ordering evidence, and compressing overflow before generation. Retrieval returns candidates; assembly decides what reaches the model. Depth starts at /context.
How do you pack retrieved chunks into a prompt?
Reserve tokens for system instructions, conversation history and generation first, then fill the remainder with retrieved evidence. Drop near-duplicates, place the strongest chunks where the model attends (edges over the middle for long prompts), and compress what still overflows. The four decisions live at /context/window, /context/dedup, /context/ordering and /context/compression.
Does a larger context window replace RAG?
Not for most retrieval-style workloads. A bigger window still costs tokens, adds latency, and suffers mid-context under-use — Liu et al. (2023; TACL 2024) measured that position effect. A Google Cloud Community summary of a long-context study notes RAG and long-context predictions were identical on about 60% of questions while diverging on cost and full-document tasks. The scored comparison is at /decisions/rag-vs-long-context.
Why does order matter in the RAG prompt?
Because models do not attend evenly across long prompts. Liu et al. (2023; TACL 2024) measured a U-shaped accuracy curve on multi-document question answering: evidence in the middle is under-used relative to the start and end. If the gold chunk was retrieved and buried mid-context, reorder on /context/ordering and diagnose the failure at /failures/lost-in-the-middle.