The Infrastructure RAG Runs On
Vector databases, embedding models, keyword indexes and graphs — each covered as what it does to retrieval quality.
RAG infrastructure is the storage, indexing, embedding and orchestration layer around retrieval, and it decides what the system can find, how safely it can return it, and how quickly that context reaches the model. This hub covers that adjacent layer only, then routes you to the infrastructure choice that changes retrieval quality next.
What is RAG infrastructure?
RAG infrastructure is the layer that stores chunks, turns them into embeddings, indexes them, filters them, and hands the retrieved context back to the model. Google Cloud’s reference architecture for Gemini Enterprise (Samantha He et al., 2025) separates that work into ingestion and serving subsystems, while Microsoft’s Azure RAG guide (Harsha Vardhan Annapureddy et al., 2026) breaks the same path into chunking, enrichment, embedding, indexing and search decisions. On this site, that layer matters only through retrieval quality: infrastructure is useful here because it changes what retrieval can find, how it can filter, and how reliably it can ground the answer.
How is RAG implemented?
RAG is implemented as an ingestion path plus a serving path, but the infrastructure choices that most change retrieval quality collapse into four buckets. Start with the bucket you are actually choosing, not with a vendor diagram that mixes every layer together.
If your problem is really about chunking, retrieval, reranking, or the scored decision of which vector database to use, go to those core pages next. This hub stays lean so the author-layer infrastructure does not cannibalize the pipeline pages it is meant to support.
Where the graph question starts
Vector search remains the default RAG foundation for unstructured content, but knowledge graphs become the better substrate when relationships are the question itself. The deeper trade-off lives at knowledge graphs for retrieval, and the architecture pattern built on that trade-off lives at GraphRAG.
What is RAG infrastructure?
RAG infrastructure is the storage, embedding, indexing, filtering and orchestration layer around retrieval. Google Cloud's Gemini Enterprise architecture guide from 2025 splits that layer into ingestion and serving subsystems, and Microsoft's Azure RAG guide from 2026 breaks the data path into chunking, enrichment, embeddings, indexing and search.
How is RAG implemented?
RAG is implemented as an ingestion path plus a serving path: documents are prepared, embedded and indexed, then user queries retrieve context that is passed to the model. On this site, the infrastructure choices that most change retrieval quality are the vector store, the embedding model, the framework abstraction and whether the retrieval problem needs a knowledge graph.
Do you need a vector database for RAG?
Most semantic RAG systems do need a vector index or vector-capable search layer, but the right choice depends on filtering, multi-tenancy, hybrid search and operations rather than on vendor popularity alone. Start with /infrastructure/vector-databases/ for the storage layer and /decisions/vector-database/ for the scored comparison.
When should RAG use a knowledge graph?
RAG should use a knowledge graph when relationship traversal, explainability, permissions, or multi-hop precision matter more than similarity alone. Glean's 2026 comparison, Paragon's 2026 guide, and Neo4j's grounding article all frame graphs as the stronger fit for relationship-heavy questions, while vector search remains the faster default for unstructured semantic retrieval.