RAG Architecture: The Pipeline and Workflow, Stage by Stage

A RAG pipeline is the stage sequence that turns source documents into grounded answers: ingest, chunk, embed and index offline; retrieve, rerank, assemble and generate online. The pipeline is how what is retrieval-augmented generation is built as a system rather than named as an idea. Weaviate’s introduction to RAG (as of September 2026) separates an ingestion stage that builds the external knowledge source from an inference stage that retrieves passages and augments the prompt before generation. Databricks’ “What is Retrieval Augmented Generation” guide walks the same offline store and online answer split. Decagon’s glossary (as of 2026) frames a RAG pipeline as the concrete implementation that wires ingest, embedding, storage, retrieval and generation into a running system. This hub maps those stages: each one decides something later stages cannot repair, and each one has a sibling page that owns the depth.

What is a RAG pipeline?

A RAG pipeline is the connected sequence of stages that retrieves passages from an external corpus and injects them into an LLM prompt before generation, so answers cite your documents rather than training weights alone. Weaviate’s introduction to RAG defines the framework as augmenting a generative LLM with additional data retrieved from an external source, citing Lewis et al. 2020 (arXiv:2005.11401) as the paper that named the method. AWS’s “What is RAG” documentation, lakeFS’s “What is a RAG Pipeline” guide, and Databricks’ end-to-end RAG workflow article all open on that retrieve-then-augment definition. Decagon draws the border cleanly: retrieval-augmented generation is the technique; the pipeline is the wired system.

The pipeline is the stage sequence. Named patterns such as naive RAG, advanced RAG, GraphRAG and agentic RAG are architectures built on top of those stages, and they live on RAG architectures, not here. Weaviate’s architecture section names three components that the stages must keep coherent: an external knowledge source, a prompt template and a generative model. Without the external store, the model is limited to parametric knowledge learned in training. With it, the model can answer from proprietary corpora, cite sources and avoid guessing when the fact was never in the training cut-off.

Every production system still has two paths even when vendors draw different boxes. The indexing path parses, cleans, chunks, embeds and writes documents into a searchable store, and that work can take minutes per document and runs asynchronously. The query path embeds each user question, retrieves candidates, usually reranks them, assembles the prompt and generates a grounded answer under a latency budget. Datarmatics’ 2026 pipeline architecture write-up calls coupling those paths, so that re-indexing takes the query API offline, the most common architectural mistake. Microsoft’s Azure Architecture Center RAG design guide names the same split as a data pipeline flow and an application flow. Keep them independent so you can change chunking or embeddings without downtime.

How does a RAG pipeline work?

A RAG pipeline works as those two independent paths in sequence. Weaviate’s how-RAG-works section labels them Stage 1 ingestion and Stage 2 inference, and under inference it names retrieval, augmentation and generation as the three online steps. The map this site uses expands that into eight stages: ingest, chunk, embed and index on the indexing path; retrieve, rerank, assemble and generate on the query path. Ranking guides group the same work differently. Databricks walks five stages from ingest to generate. Atlan names five layers including evaluation. Architecturediagram.ai uses five conceptual layers from ingestion to generation. AWS describes create external data, retrieve, augment the prompt and update the store. Decagon collapses to five named stages, chunking through generation, that still split offline versus online. The stage names vary; the offline and online split does not.

What each stage decides is narrow and consequential. Ingest decides which documents enter, how they are parsed and which metadata travels with them. Chunk decides where documents split, and that boundary is what retrieval can later find. Decagon’s glossary puts typical chunk sizes in the 200 to 1,000 token range and notes that overlap exists so information that straddles a boundary is not lost. Embed decides which vector space represents meaning, and the same model must embed queries at runtime. Index decides how vectors are stored and filtered so search is fast enough and scoped correctly. Retrieve decides which candidates come back for a query, whether dense, sparse or hybrid. Rerank decides which of those candidates enter the prompt and in what order. Assemble decides how context, instructions and the question are packed into the prompt template. Generate produces the grounded answer, including whether the model stays inside the retrieved context.

The minimum tutorial loop is six stages, load, chunk, embed, store, retrieve and generate. Production systems usually name reranking and prompt assembly as stages between retrieve and generate because that is where precision and grounding are won or lost. Databricks’ FAQ on RAG states the failure that follows when they are skipped: poor retrieval quality is the most common RAG failure mode, and a strong generator cannot compensate for irrelevant chunks. When a symptom shows up, diagnose by stage against the catalogue of RAG failure modes. A diagram of the hand-off still leaves the practical map unresolved: which sibling page owns each stage when you need depth rather than orientation.

The figure is the map at a glance. Each box still needs an owner: the hub that holds the decision and the failure that box causes when it is wrong.

What are the stages of a RAG pipeline?

The stages of a RAG pipeline are the eight decisions above, grouped into an indexing path that builds the store and a query path that answers each question. Matching the decision you are making to a stage, then opening the hub that owns that stage, is how this map stays lean. Depth lives on those pages; this hub only orients and names the failure each stage causes when it is wrong.

On the indexing path, document ingestion for RAG owns parsing, cleaning, metadata and sync, because a bad ingest seeds every retrieval failure downstream. Microsoft’s Azure RAG data-pipeline flow lists push or pull into the pipeline, then chunk, enrich with metadata, embed and persist into the search index as the per-file steps. RAG chunking strategies owns split boundaries, because wrong boundaries produce half-answers and wrong chunks. embeddings for RAG owns the vector space, because the embedding model sets the ceiling on what similarity search can match. vector indexing for RAG owns structure, updates and metadata filters, because a missing delete path is how stale documents keep ranking.

On the query path, retrieval in RAG owns dense, sparse and hybrid search, and poor retrieval is still the most common RAG failure mode named on Databricks’ FAQ. Weaviate’s inference section describes naive retrieval as similarity search over embeddings stored in a vector database such as Weaviate, Qdrant or Pinecone, and notes that production quality is heavily influenced by chunking and how content is indexed for search and filtering. Weaviate’s advanced RAG techniques post adds hybrid search, which combines similarity search with keyword search, and re-ranking with a ranker model as the common upgrades once naive dense retrieval is not enough. reranking for RAG owns the second-stage score that decides which candidates the model actually sees. generation and prompting for RAG owns prompt packing, grounding, refusal and citations, where retrieved context becomes an answer. The stage that failed is also the stage that owns the fix: re-prompting cannot repair a document that never entered the index, and a better generator cannot invent a chunk that retrieval never returned.

Vectorize’s RAG pipelines concepts page stresses the same indexing-path truth from the other side: unstructured data has to be kept current in the vector index, or the query path answers from a stale store even when retrieval looks healthy. Automating index updates is therefore part of the stage map, not an optional ops afterthought. Knowing the stage map still leaves an operating question: why teams adopt this pattern at all, and what it buys over relying on the model’s training weights alone.

Why do teams build a RAG pipeline instead of relying on the model alone?

Teams build a RAG pipeline because generative models alone cannot see proprietary corpora, post-cutoff facts or niche domain documents, and they often guess when they do not know. Weaviate’s introduction to RAG states those limits explicitly: training datasets lack niche topics and new developments beyond the cut-off, models lack access to internal databases, and when they do not know they often hallucinate in a convincing way. RAG supplies non-parametric knowledge at inference time so answers can cite sources and stay current without retraining weights.

Decagon’s glossary puts the same claim in product terms: nearly every production application that answers questions from proprietary content runs some form of RAG pipeline, from customer support agents to internal knowledge assistants to legal and medical research tools. Databricks and Cloudian both frame benefits as freshness, privacy of private corpora and reduced hallucination relative to a bare LLM. Weaviate lists real-time information retrieval, content recommendation and personal assistants that run over email, documents and notes as the common use-case shapes.

RAG is also usually cheaper to start than fine-tuning for factual grounding. Weaviate’s RAG-versus-fine-tuning section states that fine-tuning fits tone, format or domain reasoning patterns, while RAG is often the lowest-hanging fruit for accuracy and hallucination reduction because model weights do not need to be updated. Decagon treats the two as complements: a fine-tuned model for style on top of a RAG pipeline for up-to-date facts. The decision rule for when RAG is the wrong tool lives under RAG vs fine-tuning; this hub only marks the border. Benefits do not remove failure modes, and the next question is where these pipelines break once the demo corpus is gone.

Where do RAG pipelines break?

RAG pipelines break most often at the retrieve hand-off, when the index returns irrelevant or incomplete context and the generator answers that context faithfully. Databricks’ RAG FAQ states that poor retrieval quality is the most common failure mode. Decagon is blunt: the gap between a demo and production lives almost entirely in retrieval quality, because a model can only answer well from context it actually receives. Cloudian’s 2026 RAG architecture guide lists data retrieval accuracy, LLM integration and source transparency as the recurring challenges once a demo leaves the lab. Galileo’s architecture series adds hallucination detection and production monitoring as the places teams discover the break only after users complain.

Decagon names three predictable breaks. Multi-document synthesis fails when a question needs aggregation across dozens of passages and top-k returns only a handful. Stale indexes keep answering from yesterday’s policy until re-indexing, change detection or streaming ingestion catches up. Ambiguous questions return technically similar but off-target chunks, and the model sounds authoritative while missing intent. Medium’s SYNCHRONIC-scale write-up for large corpora names hybrid retrieval misses, weak reranking, unconstrained generation and missing continuous evals as the failure ladder that appears when document count grows into the millions. None of those lists invent a new stage; they describe what happens when one of the eight stages is under-specified and nobody measures it.

The practical response is stage-sorted diagnosis, not a stack of new techniques. Isolate the weak stage with the debugging procedure on the failures cluster, then open the leaf that owns that mode. When the question is how to raise answer quality across stages rather than how to name a single break, the hub for every intervention that measurably improves answer accuracy is the router. When the break is cost under load rather than wrong answers, the quantified model lives under rag cost.

Breaks also start before the first chunk is cut. Dirty PDFs, missing metadata and unstructured dumps that never became clean documents look like retrieval bugs later. That upstream work is rag data preparation, and it is the cheapest place to prevent a class of failures that no reranker can fix. Once breaks are pinned to stages, the remaining question is how to evaluate the pipeline so changes are measured rather than guessed.

How do you evaluate a RAG pipeline?

You evaluate a RAG pipeline by scoring retrieval and generation separately, then checking that the end-to-end answer still matches the ground truth. Weaviate’s evaluation section states that RAG needs both component-level and end-to-end checks: retrieval accuracy and relevance on one side, faithfulness and correctness on the other, plus answer semantic similarity as a simple joint signal. Decagon’s glossary says a serious pipeline measures retrieval precision, retrieval recall, answer faithfulness and answer helpfulness on a fixed set of representative queries, and that without evals every change is a guess.

Microsoft’s Azure Architecture Center RAG series treats evaluation as a structured experiment loop across preparation, chunking, enrichment, embedding, retrieval and language-model end-to-end phases, with documentation of hyperparameters and aggregated results. Weaviate names RAGAS as a popular framework that offers metrics for retrieval relevance, generation quality and faithfulness without requiring human-labeled data for every run. Depth on metrics, tools and online monitoring lives under how to evaluate a RAG system; this hub only locks the rule that you measure before you stack techniques.

Evaluation also decides when a fixed pipeline is no longer enough. Microsoft’s standard-versus-agentic distinction says a fixed orchestrator works when queries map to a single search against a single index, and that multistep reasoning, dynamic source selection and query decomposition at runtime push toward agentic RAG where retrieval is a tool the agent invokes. Weaviate’s techniques section names advanced RAG, agentic RAG and Graph RAG as the common extensions beyond vanilla similarity search. Those patterns are owned by the architectures cluster. Knowing how to measure still leaves the start question: build a working loop, or diagnose an existing one.

How do you start building or diagnosing a RAG pipeline?

You start by deciding whether you need a runnable baseline or a diagnosis of a system that already answers wrong. If you need working code with the choice at each stage stated and measured, open the rag pipeline build tutorial and implement the six-stage loop before you add reranking or agentic control. NVIDIA’s RAG 101 pipeline article follows that same build-first shape: document ingestion, pre-processing, embeddings, vector storage, then querying with an LLM. Weaviate’s implementation section shows how to wire Weaviate with common orchestration libraries once the stage choices are fixed, and keeps the vector store as the default retrieval backend in those recipes.

If something already returns the wrong answer, do not re-chunk and hope. Measure retrieval and generation separately, map the weak metric to a stage, and change only that stage before stacking the next lever. That measure-diagnose-intervene order is what the accuracy hub encodes, and it is what Cloud Google’s RAG optimization posts treat as Step 1 before any experiment list. Vector stores such as Weaviate, Qdrant and Pinecone change what retrieve and index can do; they do not replace the need to know which stage failed. Galileo’s production monitoring guidance and Cloudian’s scale practices both treat independent measurement of retrieval and generation as the difference between a durable pipeline and a demo that only worked once.

Operating concerns sit beside build and diagnosis. Corpus cleaning before the chunker is data preparation. Dollars per million documents and per thousand queries are cost. Effect-ranked interventions across chunking, retrieval, reranking and evaluation are accuracy. Named architecture patterns beyond the stage sequence live under architectures. Orchestration depth belongs under infrastructure once you already know which stage you are changing. Vectorize’s freshness warning still applies at start-up: if the index is not updated when documents change, the first query path you ship will answer from stale chunks. The pipeline hub’s job is to keep those borders clear so the next click matches the decision you are actually making.

A RAG pipeline is offline indexing plus online answering, eight stage decisions, and a hand-off at retrieve where most wrong answers are born. Weaviate, Databricks, Decagon, AWS and Microsoft’s architecture guidance all describe that two-path shape even when they draw different boxes. Build the six-stage tutorial loop when you need code, prepare the corpus before you chunk, measure cost and accuracy as operating questions, evaluate retrieval before you blame the model, and diagnose by stage when answers fail. The sibling hubs own the depth; the stage borders and the failure each stage causes remain on this map of the workflow.