What Is Retrieval-Augmented Generation? RAG Explained
RAG answers from documents fetched at query time instead of weights alone. What each stage does, what it costs, and what it does not fix.
Retrieval-augmented generation (RAG) is a technique that fetches relevant documents at query time and hands them to a language model as context — so the answer is grounded in your data, not in the model’s frozen training weights alone. It is a pipeline with moving parts — each one measurable, each one able to break. This page defines it, shows how it works, and is the map to every part in depth.
How does RAG work?
In two phases. Index-time, done once: your documents are split into chunks, each chunk is embedded into a vector, and the vectors are stored in a search index. Query-time, on every question: the question is embedded, used to retrieve the top-k most similar chunks from that index, and sent together with the question to a language model that generates a cited answer. The diagram is the whole territory in one view — every box is a cluster that goes deep below.
Why use RAG — what problem does it solve?
A plain language model answers only from its frozen training weights. So it cannot see your private documents, it goes stale the moment the world changes, and when it does not know something it tends to invent a confident answer. RAG fixes the access problem — it grounds the answer in fetched, current, private data with a citation trail. Three wins follow:
- Current and private knowledge without retraining — add a document to the index and it is answerable immediately; no model retrain.
- Source attribution — the retrieved chunks are the citations, so an answer can be traced back to where it came from.
- Fewer ungrounded hallucinations — giving the model the real text to quote reduces the guessing. Note the word reduces — it does not eliminate, which the next section is honest about.
What are RAG’s benefits and limitations?
Both sides, in one place — a page that only lists benefits is selling something. The benefits are real: no retraining to add knowledge, built-in source citations, current and domain-specific data, and lower cost than fine-tuning when the goal is injecting facts. The limitations are just as real, and RAG does not hide them:
- Retrieval can fetch the wrong chunk. Garbage in, garbage out — if the retriever pulls the wrong passage, the model answers confidently from it. This is the most common failure; its anatomy is at the wrong-chunk failure.
- It adds latency and infrastructure. A vector store, an embedding step and a retrieval hop sit in front of every answer.
- It does not fix reasoning or a weak base model. RAG supplies knowledge, not intelligence — a model that reasons badly still reasons badly with better context.
- It reduces but does not remove hallucination. A wrong retrieved chunk produces a wrong grounded answer. Whether yours is any good is a measurement question — see evaluation.
The honest summary: RAG is an access fix, not an intelligence fix. That framing is what keeps the rest of this site from overselling it.
RAG or fine-tuning — which do you need?
They solve different problems, so it is rarely either/or. RAG injects knowledge — facts the model should look up at answer time. Fine-tuning changes behaviour — tone, format, or a skill baked into the weights. The one deciding condition: if the answer depends on data that changes or is private, reach for RAG; if you need the model to act differently, fine-tune. The scored either/or/both call — with the cost comparison — is at RAG vs fine-tuning.
Where does each part of RAG go deep?
Nine clusters cover the whole pipeline. Each is an authority hub with a page for every part it names — start wherever your problem lives.
Or start by what you are trying to do:
Or take it into the real world — apply it, ship it, and choose between the alternatives:
Is RAG the same as fine-tuning?
No. RAG injects knowledge — facts the model looks up at answer time from an external index — while fine-tuning changes behaviour, baking tone, format or a skill into the weights. If the answer depends on data that changes or is private, use RAG; if you need the model to act differently, fine-tune. They are often combined. The full comparison is at /decisions/rag-vs-fine-tuning.
Does ChatGPT use RAG?
The base language model does not — it answers from its training weights. But when ChatGPT browses the web or searches connected files, it is doing retrieval-augmented generation in effect: fetching documents and answering from them. So the model alone is not RAG; the product feature that reads outside sources is.
Does RAG stop hallucinations?
It reduces them, but does not eliminate them. Giving the model real retrieved text to quote cuts down ungrounded guessing. But if retrieval fetches the wrong passage, the model produces a confident answer grounded in the wrong source — a wrong-chunk failure. That is why measuring faithfulness matters even with RAG in place.
Do you need a vector database for RAG?
Usually, because retrieval works by finding chunks whose embeddings are most similar to the question, and a vector database is what stores and searches those embeddings at scale. Small prototypes can use an in-memory index, but production RAG typically runs a real vector store. The options are compared under /infrastructure.
What is RAG used for?
Grounded question-answering over private or current documents: support chatbots that cite your knowledge base, internal search that answers in prose, and assistants that must reference specific, up-to-date sources rather than guess. Anywhere an answer must come from your data with a citation trail, RAG is the usual technique.