How to Build a RAG Pipeline From Scratch: A Python Tutorial

A RAG pipeline is six stages run in two phases: load, chunk, embed, and store at index time, then retrieve and generate at query time. Index time builds a searchable collection once from your documents. Query time answers each question against that collection by fetching evidence and prompting a language model to answer only from what was retrieved. Ranking tutorials for “build a RAG pipeline” almost all walk those stages, from Hugging Face’s from-scratch vector list through DigitalOcean’s end-to-end guide and DataCamp’s LangChain plus FastAPI tutorial, but they default chunk size, embedding model, top-k, and store class without stating the trade-off. The build below pins versions, states each choice, ships an evaluation harness so you measure your own pipeline, and treats hybrid search or reranking as gated additions rather than defaults. If you are still deciding whether RAG is the right tool against fine-tuning, start at RAG vs fine-tuning before you wire the stages.

How do you set up the environment and pin versions?

You set up a first RAG build by installing a pinned client stack and exporting the API keys that stack will call. Pinning is the difference between a tutorial that still runs next quarter and one that silently breaks when a major version ships. Weaviate’s documentation (as of September 2026) installs the Python client with pip install -U “weaviate-client[agents]”; PyPI reported weaviate-client==4.23.1 on 18 September 2026. For the learn path that keeps an in-process store beside the Weaviate path, the packages below are pinned so the snippets paste and run as written.

Create a virtual environment, install the pins, and put secrets in the environment rather than in the repo. OpenAI is used here for a hosted embedding and chat model because most ranking tutorials do the same; a local embedding path is shown in the embed section. If a newer release is out when you read this, upgrade deliberately and re-run the eval harness after the bump, not by letting an unpinned install drift.

pip install weaviate-client==4.23.1 \
  langchain==0.3.14 langchain-openai==0.2.14 langchain-community==0.3.14 \
  langchain-text-splitters==0.3.4 langchain-chroma==0.1.4 \
  chromadb==0.5.23 ragas==0.2.9 datasets==3.2.0

export OPENAI_API_KEY="sk-..."          # hosted embed + chat
export WEAVIATE_URL="https://..."       # Weaviate Cloud REST URL when using the server path
export WEAVIATE_API_KEY="..."           # admin key from the Weaviate Cloud console

Nimble’s eight-step RAG pipeline guide and DigitalOcean’s end-to-end tutorial both open with dependency install before any chunking code, which is the right order: a broken import is not a retrieval failure. DataCamp’s LangChain and FastAPI tutorial also loads environment variables before the first document load. With the environment fixed, the first content decision is how documents enter the pipeline and how they are split.

Pinning also freezes the evaluation dependency. The harness below uses ragas==0.2.9 because that is the API surface the paste block calls; PyPI also listed newer Ragas releases on 18 September 2026, and those releases change import paths. When you upgrade Ragas, re-pin and re-run the same labelled rows before you trust a score delta. The same rule applies to weaviate-client: the 4.23.1 pin matches the collection API shown in the store section, and a major client bump should be treated as a deliberate migration.

How do you load and chunk your documents?

You load raw documents into memory, then split them into overlapping passages before any embedding runs. Chunk size and overlap are the two knobs that decide whether a fact stays attached to the sentences that make it usable. Too large and the answer is buried in noise the model has to wade through. Too small and a sentence is severed from the paragraph that defines its terms. DigitalOcean’s end-to-end RAG pipeline tutorial, GeeksforGeeks’ open-source LLM build, DataCamp’s LangChain walkthrough, and Hugging Face’s from-scratch blog all treat load-then-chunk as the first content stages; Dev.to’s production RAG write-up by Gautam Vhavle spends the longest narrative on chunking because that is where most first builds go wrong.

The defaults below use 500 characters and 50 characters of overlap as a starting point to measure, not as a law. They are character counts in LangChain’s RecursiveCharacterTextSplitter, not tokens; if your tokenizer averages differently, restate the same choice in tokens and re-score. Comparing fixed, semantic, and recursive chunkers in depth belongs under chunking strategies. When retrieval returns a near-miss that was cut mid-thought, the failure mode is the wrong chunk, not a missing document.

from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter

docs = TextLoader("handbook.txt").load()

splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,      # choice: passage length in characters
    chunk_overlap=50,    # choice: carry-over so facts are not severed at the seam
)
chunks = splitter.split_documents(docs)
print(f"{len(docs)} document(s) -> {len(chunks)} chunks")

Nimble’s guide inserts a cleaning step between collection and chunking; stripping boilerplate and repeated headers before you split is owned by preparing data for RAG and document cleaning, not by the first paste block above. Once passages exist, the next choice is which embedding model turns them into vectors.

PDF layout, table structure, and OCR error rates change what the splitter sees. Those ingestion paths are separate nodes under document ingestion for RAG, including parsing PDFs and tables and structured data. A first handbook build on a plain text file is enough to learn the six stages; enterprise corpora should not skip the ingestion siblings.

Which embedding model should you use?

You should pick an embedding model by language coverage, dimension, latency, and cost on your own retrieval task, not by brand ranking alone. DigitalOcean names text-embedding-3-large, all-MiniLM-L6-v2, bge-large-en, and e5-large-v2 as common options with different trade-offs in performance, cost, and deployment. Joey O’Neill’s Medium introduction embeds with OpenAI’s text-embedding-3-large into an in-process store. Dev.to’s production write-up lists OpenAI text-embedding-3-large beside self-hosted sentence-transformers. The decision is dimensions versus cost versus recall on your labelled queries.

A hosted model is the least operational work and sends text to a provider. A local sentence-transformers model has no per-token API charge and keeps text on your hardware at the price of CPU or GPU time. OpenAI publishes current embedding prices on its API pricing page; read the live rate for the model you choose before you budget, because rates change and a memorised figure goes stale. Poor model choice shows up later as systematic near-misses; depth on what an embedding encodes and how to compare models lives at embeddings in RAG and how to choose an embedding model.

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
# local alternative:
# from langchain_huggingface import HuggingFaceEmbeddings
# embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")

GeeksforGeeks’ open-source build pulls Hugging Face embeddings with an access token; that is the same local-or-hub choice stated above, not a different stage. With vectors about to exist, you need a store that can answer nearest-neighbour queries at index and query time.

Dimension truncation and Matryoshka-style shorter vectors are later cost levers, not first-build requirements. If storage pressure appears after the eval passes, measure recall under truncated dimensions on embedding dimensions rather than guessing a smaller model. Domain vocabulary that still misses after hybrid search is live belongs under domain adaptation of embeddings, not under a silent swap of the first embed call.

How do you store and query the vectors?

You store chunk vectors in a system that supports nearest-neighbour search so query embeddings can find nearby passages. For a production collection that will grow, take filters, or run hybrid search in one request, a server database is the usual graduation. Popular vector stores for that job include Weaviate, Pinecone, Qdrant, and Milvus. Weaviate’s quickstart (as of September 2026) creates a collection, imports objects, runs similarity search, then runs RAG with a generative model against the same cluster. For a first laptop build only, after you understand the Weaviate collection path, an embedded in-process store such as Chroma or FAISS still runs without a server and is useful while you learn the six stages.

State the choice without pretending they are the same product. Embedded stores minimise ops for a small corpus and a single process. A Weaviate collection (Cloud or self-hosted) gives you persistence, filters, and a path to BM25F plus vector fusion in one hybrid query when dense-only retrieval later fails on exact terms. Joey O’Neill’s Medium tutorial and several orchestration guides default to an in-process Chroma folder; Hugging Face’s from-scratch post implements an in-memory list of (chunk, embedding) tuples. Those paths are fine for learning. Production usually leaves the in-process folder behind for Weaviate or another server store.

from langchain_chroma import Chroma

store = Chroma.from_documents(
    documents=chunks,
    embedding=embeddings,
    persist_directory="./rag_index",
)

import os
import weaviate
from weaviate.classes.init import Auth
from weaviate.classes.config import Configure, Property, DataType

client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ["WEAVIATE_URL"],
    auth_credentials=Auth.api_key(os.environ["WEAVIATE_API_KEY"]),
)

if client.collections.exists("Handbook"):
    client.collections.delete("Handbook")

client.collections.create(
    name="Handbook",
    properties=[Property(name="text", data_type=DataType.TEXT)],
    vectorizer_config=Configure.Vectorizer.text2vec_openai(
        model="text-embedding-3-small"
    ),
)
# import chunks via collection.data.insert_many(...); then query with collection.query.near_text

Choosing among server products in depth is vector databases for RAG. Multi-tenant isolation and delete paths are later concerns owned by indexing and ingestion siblings. The remaining paste blocks keep a short in-process retriever so the query path stays readable; swap that retriever for a Weaviate near-text query when you leave the laptop. Retrieval is the next stage either way.

Index structure still matters once the store is chosen. HNSW parameters, IVF partitions, and quantization trade recall for memory and latency; those knobs are owned by indexing for RAG and should not be tuned until a baseline retrieve-and-generate path has scores. A first build that never measures recall will mistune ef and call it progress.

How do you retrieve the right chunks?

You retrieve by embedding the question with the same model used at index time and fetching the top-k nearest chunks from the store. Top-k is a recall-versus-cost dial: too low and gold evidence never enters the prompt; too high and you flood the model with irrelevant text and pay for the extra tokens. DigitalOcean, DataCamp, GeeksforGeeks, and Medium’s LangChain introduction all expose a retriever or similarity search with an explicit k. Start at k=4 for a small handbook corpus and move it only when the eval harness says recall is the bottleneck.

When dense retrieval keeps missing exact terms such as IDs, error codes, or rare tokens that appear verbatim in the documents, raising k is usually the wrong first repair. Add a lexical channel through BM25 and fuse it under hybrid retrieval. Weaviate’s hybrid search documentation (as of September 2026) combines vector search with keyword BM25F in one query, with alpha of 1 as pure vector and alpha of 0 as pure keyword, and Relative Score Fusion as the default fusion method from Weaviate v1.24. Dataquest’s from-scratch post describes the related vocabulary gap and multi-query expansion when the user’s phrasing never matches the document’s wording; that is a retrieval design change, not a larger k.

retriever = store.as_retriever(search_kwargs={"k": 4})   # choice: k

hits = retriever.invoke("How do I reset my password?")
for h in hits:
    print(h.page_content[:80])

Retrieval quality without generation still leaves you with passages, not an answer. The next stage is assembling those passages into a grounded prompt and calling the model.

Context window budgeting, compression, ordering, and deduplication decide which of the retrieved passages survive into the prompt. Those assembly concerns are owned by the context assembly cluster and matter once top-k regularly returns more text than the model can use. For a first handbook build with k=4, assembly is mostly concatenation; for long-document corpora it becomes its own stage.

How do you assemble the prompt and generate the answer?

You assemble a prompt that places the retrieved chunks in a context block and instructs the model to answer only from that context, refusing when the evidence is missing. That instruction is what separates RAG from an unconstrained chatbot: generation is constrained to your documents rather than to parametric memory alone. Hugging Face’s from-scratch guide, DigitalOcean’s generation section, DataCamp’s QA chain, and Joey O’Neill’s Medium post all show a context-plus-question template; Dataquest’s from-scratch narrative explicitly walks the case where better prompting stops a hallucination when the docs do not cover the question.

Keep temperature low for factual handbook questions so decoding stays stable across eval runs. Weaviate’s RAG documentation (as of September 2026) describes a RAG query as search plus a generative prompt in one request, with single-prompt and grouped-task shapes; the pasteable block below keeps the same contract beside an in-process retriever, while a Weaviate generative query can collapse search and prompt into one request when you are ready. Ungrounded generation is the failure cluster under why RAG systems fail, and citation-level attribution is a later generation concern owned by the generation cluster.

Refusal behaviour when context is empty is part of grounding, not a separate product feature. Dataquest’s from-scratch narrative shows the model inventing an answer when the docs do not cover the question, then recovering after the prompt requires abstention. Teaching abstention thresholds in depth is teaching a RAG system to say it does not know; the one-line instruction in the template is the minimum that keeps a first build honest.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_template(
    "Answer the question using ONLY the context below. "
    "If the context does not contain the answer, say you don't know.\n\n"
    "Context:\n{context}\n\nQuestion: {question}"
)
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)

def answer(question: str) -> str:
    hits = retriever.invoke(question)
    context = "\n\n".join(h.page_content for h in hits)
    msg = prompt.format_messages(context=context, question=question)
    return llm.invoke(msg).content

print(answer("How do I reset my password?"))

A pipeline that answered three manual questions is still a demo until you can score changes. Evaluation is the stage that turns every earlier choice into a measurement.

How do you know the pipeline actually works?

You know the pipeline works when a labelled eval set reports faithfulness and retrieval metrics that move when you change one stage at a time. “It looked right on three questions I tried” is not a score. DigitalOcean includes an evaluating-RAG-system-performance section in the same end-to-end tutorial. Nimble’s step 7 pairs orchestration with evaluation and monitoring. Dataquest’s from-scratch post ends in a diagnostic checklist after walking five failure modes. Guides that stop at generation leave the reader without a way to measure chunk size, embedding choice, or top-k.

Build a small set of question, ground-truth, and expected-context rows from documents you control. Run a harness that reports faithfulness (did the answer stick to retrieved context) and context precision or recall (did retrieval fetch the right chunks). Metric definitions and tool comparisons live at how to evaluate a RAG system and evaluation tools; Ragas is one harness among several, shown here because several production write-ups name it and the API is short enough to paste.

from ragas import evaluate
from ragas.metrics import faithfulness, context_precision, context_recall
from datasets import Dataset

rows = [{
    "question": "How do I reset my password?",
    "answer": answer("How do I reset my password?"),
    "contexts": [h.page_content for h in retriever.invoke("How do I reset my password?")],
    "ground_truth": "Use the 'Forgot password' link on the sign-in page.",
}]

report = evaluate(
    Dataset.from_list(rows),
    metrics=[faithfulness, context_precision, context_recall],
)
print(report)

Once scores exist, the next question is whether denser retrieval machinery is actually required, or whether the dense-only path already passes.

Synthetic question generation can expand a tiny eval set, but synthetic rows still need human spot checks before you trust a gate. Depth for building and refreshing labelled sets lives under test sets for RAG and synthetic evaluation data. A harness on five hand-written handbook questions is enough to start; shipping without any labelled rows is how silent regressions land.

When should you add hybrid search or reranking?

You should add hybrid search or reranking only when the eval shows a specific miss that those stages fix. Bolting them on by default buys latency and cost for no measured gain. Dataquest’s from-scratch narrative reaches multi-query expansion after a vocabulary gap appears. Dev.to’s production post discusses advanced retrieval after a minimal FAISS path works. Competitors often enable every enhancement in the first listing; the gain here is the gate.

Three gates cover most first production systems. Add hybrid search (BM25 beside dense) when exact terms, IDs, or error codes miss under embeddings alone; mechanism depth is combining BM25 and vector retrieval. Add reranking when recall is fine (the gold chunk is in the top 20) but not ranked first; a cross-encoder reorders the shortlist under reranking. Add multi-query or rewriting when questions are phrased far from source wording so one query vector misses; depth sits with query rewriting. If faithfulness and context recall already pass on your labelled set, add none of them yet.

Weaviate, Qdrant, and Pinecone all expose native hybrid shapes that remove client-side fusion glue; Weaviate leads that list when naming stores with a hybrid API, and the fusion defaults still need an eval before you trust them. Latency and cost of each added stage belong in the production bill, which is the closing practical question of a first build.

If the gold chunk never enters the candidate set at all, the failure is not ranking and not fusion. Prove absence on the missing-document sequence under when RAG misses the document you know is there, then fix ingestion or ACL filters before adding rerankers. Enhancements cannot retrieve a document that was never indexed.

Should you build from scratch or use a framework?

Build from scratch when the goal is to see every stage and own the control flow; reach for a framework when the goal is shipping a standard shape with maintained plumbing. Dataquest’s ranking post is explicitly “no frameworks” and walks hallucinations and retrieval failures by hand. DataCamp and Joey O’Neill’s Medium guide use LangChain end to end. GeeksforGeeks introduces LangChain as the orchestration layer around open-source LLMs. The six stages are the same either way; the trade is control for speed.

A common middle path is to borrow loaders and splitters while wiring retrieve-and-generate yourself, which is how the paste blocks here are shaped. Framework abstractions fail in the same places custom code fails: bad chunks, weak embeddings, ungrounded prompts. Debugging through an opaque chain without stage-level scores is slower than debugging six named functions with a harness. Framework-specific deep builds stay under the RAG pipeline hub; the trade table that follows is enough to choose a path for a first system.

Serving the pipeline behind an HTTP API, as DataCamp does with FastAPI after the chain works, is a production concern rather than a first-build stage; route that depth to production deployment once the eval passes. The last question a builder asks before leaving the six stages is what the recurring bill looks like.

Build-versus-buy for the whole platform is a different decision from framework-versus-scratch for the wiring. If the question is whether to operate your own retrieval stack at all, that comparison lives with platform decisions rather than with the paste blocks here. For learning the stages, scratch or thin wiring wins; for shipping a standard chatbot shape quickly, a maintained framework wins on calendar time.

What does it cost to run in production?

A production RAG bill has three recurring lines tied to stages you already built: embeddings at index time, generation per query, and the vector store host once you leave an embedded database. DigitalOcean’s cost and latency section and Dev.to’s production cost notes both put generation as the usual dominant per-query line. Embeddings are paid when documents are indexed and again when they change. The store host is a standing cost for Weaviate Cloud or any managed vector service; an in-process Chroma folder has no host line until you need high availability.

Latency is dominated by the generation call for most handbook workloads, not by ANN retrieval, once the index fits in memory or a nearby region. Levers map back to stages: cache repeated answers, batch embeddings at index time, drop to a smaller chat model when faithfulness still passes, and lower top-k so fewer tokens enter the prompt. Full cost models per million documents and per thousand queries live at what a RAG system actually costs; incremental re-embedding when documents change is incremental indexing. Monitoring and regression gates after you ship are evaluation and production siblings, not part of the first paste.

Dev.to’s production write-up quotes illustrative generation prices for hosted GPT-4 versus a self-hosted Llama-3 8B path as a motivation to measure; treat those figures as that author’s examples, not as current public rates, and re-read your provider’s pricing page before any budget. The structural claim that survives rate changes is that generation usually dominates per-query spend while embeddings dominate when the corpus churns.

A first RAG build is complete when the six stages run under pinned versions, each choice is stated, the eval harness scores your own questions, and hybrid or reranking wait on a measured miss. Which stage to deepen next depends on which scores move: chunking, embeddings, retrieval, or generation.