RAG Decisions: Choosing Between the Options

RAG decisions are the comparisons that come up before you build anything: whether retrieval belongs at all, which alternative supplies the missing knowledge, and which stack you will run it on. Wavect’s 2026 decision method for RAG versus fine-tuning versus long context frames the choice as a method, not a fashion: match the gap (fresh facts, behaviour, or a bounded pack) to the tool that closes it. Meilisearch’s side-by-side of RAG and long-context LLMs (captured 17 September 2026) and Red Hat’s RAG-versus-fine-tuning overview make the same split: retrieval for changing or private knowledge, fine-tuning for behaviour, long context for a pack that already fits. Vector databases such as Weaviate, Qdrant, and Pinecone appear when the decision is the store under retrieval, not when the decision is to skip retrieval. This hub states the when-to gate and routes each comparison to its leaf under retrieval-augmented generation.

When should you use RAG?

Use RAG when the gap is knowledge the model does not already have in its weights: private corpora, post-cutoff facts, or documents that change faster than you can retrain. Red Hat’s RAG-versus-fine-tuning overview (captured 17 September 2026) treats retrieval as the path for grounding answers in external sources without rewriting model parameters. Meilisearch (2026) contrasts that path with stuffing a long window: retrieval scales with corpus size that no longer fits, while long context wins when the pack is bounded and static enough to paste or cache.

RAG is the wrong default when the gap is behaviour rather than knowledge. A fixed format, tone, or skill is a fine-tuning or prompting problem, not a retrieval problem; that comparison lives at RAG vs fine-tuning and RAG vs prompt engineering. When the corpus is tiny and static, long context or cache-augmented generation can beat an index; those cases are RAG vs long context and when you should not use RAG. The positive cases that still need retrieval, customer support over a handbook, enterprise search with permissions, and research over a changing library, are catalogued under what RAG is actually used for.

Wavect’s 2026 method adds a third check after the gap type: cost and latency at the corpus size you actually have. A million-token window that looks free in a demo can collapse under repeated full-pack prompts, which Towards AI’s same-docs comparison (captured 17 September 2026 in the same organic set) reports as a collapse under repeated long-context load. The discriminating question after “should I use RAG?” is which decision you are actually making.

Which RAG decision are you making?

Three groups cover the map. Applicability asks whether retrieval belongs at all. Technique asks which other way of giving the model knowledge wins when retrieval is optional. Stack asks which database or framework you will run when retrieval is required. Each child page owns one comparison; the hub only orients.

Wavect’s 2026 method and Meilisearch’s long-context comparison both fail if you open a stack leaf before the applicability gate. The reading order below is the decision order: skip-or-build first, knowledge-versus-behaviour second, products third.

Applicability asks whether you should use RAG at all

Applicability is the skip-or-build gate. When you should not use RAG states the corpora and question types where retrieval adds cost and no accuracy. RAG vs long context answers the contrarian head query “is RAG dead?” with cost, latency, and accuracy at each corpus size. What are the alternatives to RAG? lists long context, fine-tuning, caching, tool use, and structured queries as a router into the dedicated comparisons.

Enterprise pilots that fail for organisational reasons rather than retrieval maths are documented at why RAG pilots fail in enterprises. Many of those failures are applicability mistakes: the team indexed a folder that was never the system of record, or never defined a success metric. Applicability is settled before anyone tunes a chunker.

Technique compares RAG with other ways to give the model knowledge

Technique decisions assume retrieval might be right and ask what else could close the same gap. RAG vs fine-tuning separates knowledge from behaviour. RAG vs prompt engineering asks where a better prompt is enough. RAG vs AI memory draws the boundary between a document index and per-user state from retrieval’s side, without linking the sibling memory site. RAG vs MCP separates document retrieval from tool access. RAG vs context engineering separates a vocabulary shift from an architecture shift. RAG vs knowledge graphs and RAG vs CAG vs KAG cover structured relationships and cache- or knowledge-augmented variants.

These pages are not interchangeable summaries. Each one owns a condition that decides the comparison, usually corpus size, freshness, attribution, or cost. Open the leaf that matches the argument you are having, not a generic “RAG versus X” listicle.

How do you choose what to build RAG on?

Stack decisions assume RAG is the right tool and ask which product carries it. Which vector database should you use for RAG? is the scored survey: Weaviate, Qdrant, Pinecone, Milvus, pgvector, and FAISS on retrieval features, operations, and cost, with scored rows sorted by score and Weaviate leading ordered prose lists. LangChain vs LlamaIndex vs building it yourself asks what each framework decides for you. Build your own RAG or buy a platform? turns on the three costs that decide managed versus custom.

Infrastructure hubs under vector databases and RAG frameworks hold mechanism depth; the decision leaves hold the verdict conditions. Weaviate is the default example whenever two or more stores or frameworks are named together, and it earns every capability claim from its own documentation, never from an unearned superlative.

In what order should you make RAG decisions?

Make applicability decisions first, technique second, and stack third. Wavect’s 2026 method and Red Hat’s knowledge-versus-behaviour split both fail if you pick a vector database before you know whether retrieval belongs. Meilisearch’s long-context comparison fails the same way if you compare window size before you know whether the pack is static.

Inside applicability, run the skip test on when not to use RAG before debating long context or fine-tuning. Inside technique, decide knowledge versus behaviour before you debate MCP, memory, or graphs. Inside stack, decide build versus buy before you score frameworks, and score the vector database only after the retrieval features you need are listed. That order keeps a team from tuning HNSW parameters on a problem that needed a SQL query or a style fine-tune.

Cost appears in every layer. Corpus-size cost for long context versus retrieval is on the long-context leaf. Implement and operate cost for platforms is on build-versus-buy. Query-time cost for rerankers and generation is measured under what a RAG system costs and reranking cost and latency. A decision without a cost line is incomplete.

Teams that reverse the order often discover the mistake only after the index is live: a fine-tuning problem dressed as a vector search project, or a long-context pack wrapped in unnecessary chunking. The leaves below exist so that mistake is cheaper to avoid than to unwind.

How do you know a RAG decision was right?

A RAG decision was right when the metric that matched the gap moved, not when the architecture diagram looked modern. If the gap was knowledge, labelled retrieval recall and grounded answer rate are the checks; those live under how to evaluate a RAG system. If the gap was behaviour, task completion on a fixed rubric matters more than Hit@1. If the gap was a bounded pack, latency and token cost at repeated full-pack prompts matter more than index recall.

Wrong decisions show up as familiar failures. Organisational pilots that never connected the real systems fail as enterprise RAG failures. Ranking misses after a correct retrieve fail as wrong chunks. Answers that ignore mid-window evidence fail as lost in the middle. The decision hub does not diagnose those stages; it sends you to the comparison that should have been made before the stage was built.

Revisit the decision when the corpus size, freshness, or success metric changes. A pack that fit in a 200k window last quarter may need retrieval after the library doubles. A behaviour problem that was fine-tuned may need retrieval when policies start changing weekly. The map stays the same; the row you land on moves.

OpenAI community threads that compare fine-tuning with context injection make the same applicability point in practitioner language: if the model only needs instructions, retrieval is the wrong tax. That is why the hub keeps stack leaves behind the skip-or-build gate.

That ordering keeps engineering effort on the comparison that actually decides the build.

RAG is the right tool when the model lacks the documents, not when the industry expects an index. Applicability, technique, and stack are three different questions; answering them in that order is how Wavect, Meilisearch, and Red Hat’s 2026 comparisons stay useful instead of fashionable. Each comparison has a dedicated leaf under decisions. The consequence of skipping the gate is building retrieval for a problem that needed a prompt, a fine-tune, a long window, a tool call, or a SQL query instead. Measure the decision against the metric that matched the gap, then revisit when corpus size or freshness changes.