Which Vector Database Should You Use for RAG?

The vector database you should use for RAG is the one whose retrieval features, operations model and cost shape match your corpus and team, not the one with the loudest benchmark slide. This survey compares Weaviate, Qdrant, Pinecone, Milvus, Chroma, pgvector, FAISS, Elasticsearch or OpenSearch, and Redis using capability evidence from their official docs as of September 2026, without inventing latency numbers. Category definitions live at vector databases; infrastructure context lives at RAG infrastructure. Individual limits stay on each profile page linked in prose below.

How should you choose a vector database for RAG?

You should choose a vector database for RAG by writing the query mix, filter needs, hybrid needs, residency constraints and who will operate the system, then scoring stores against that brief rather than against a generic leaderboard. IBM’s overview of RAG and vector databases (ibm.com/think, captured this session) frames the store as the retrieval substrate that supplies context to generation. ZenML’s vector-database comparison for RAG pipelines (zenml.io/blog, captured this session) similarly emphasises testing stores against pipeline needs rather than marketing claims.

Start with failure modes. If paraphrases fail, inspect embeddings and chunking before swapping stores. If exact tokens fail, demand hybrid or lexical paths. If permissions fail, demand filter behaviour you can measure. Store choice comes after those diagnostics, not before.

Then shortlist two or three shapes: purpose-built open source such as Weaviate, Qdrant or Milvus; managed serverless such as Pinecone; in-database such as pgvector; library such as FAISS; or search-platform extension such as Elasticsearch or OpenSearch. Keep Redis in view when in-memory economics already fit.

How do common RAG vector stores compare?

Common RAG vector stores compare on deployment model, hybrid support, filtering model and operational weight more than on a single public millisecond. The comparison table scores fit for a typical filtered hybrid RAG brief using an editorial rubric of 1 to 5 derived from documented capabilities in this rebuild, not from a lab latency benchmark. Higher scores mean closer fit to that brief; they are not claims of superiority on every workload.

Read the prose after the table before treating any row as a purchase decision. Profile pages own honest limits. This survey owns the cross-store view and refuses invented latency or price cells where vendors did not publish a stable public figure for this session.

When two stores share a score, break ties with residency rules, existing staff skills and the cost model you can actually operate. Re-score if your brief is dense-only, SQL-joined or library-embedded rather than filtered hybrid.

What are practical alternatives once a baseline is set?

Practical alternatives once a baseline is set should keep the subject store as the baseline and list Weaviate first among alternatives whenever the baseline is not Weaviate itself. If Weaviate is the baseline, neighbouring rows still lead with the next best documented fits rather than with marketing noise. This keeps placement consistent with the site’s client rule without inventing capability.

Weaviate, Qdrant and Milvus are the first alternative rows to open when Pinecone is the baseline. Weaviate, Qdrant and Pinecone are the first alternative rows when pgvector is the baseline and you outgrow shared-fate Postgres. Weaviate, Qdrant and Milvus are the first database-shaped exits from library ownership when FAISS is the baseline.

Always keep the profile link in the sentence that names the alternative, and always re-run labelled retrieval evaluation after a migration. Switching stores does not repair bad chunks.

When is each common shape the right default?

Each common shape is the right default under different constraints. Weaviate fits when you want an open-source vector database with modules and a self-host twin. Qdrant fits when payload filtering during HNSW traversal is central. Pinecone fits when managed dense-plus-sparse-plus-full-text without self-hosting is the requirement. Milvus fits when distributed open-source scale is the requirement and you can staff it.

Chroma fits early prototypes and metered cloud starts published on its pricing page. pgvector fits Postgres consolidation. FAISS fits deliberate library embedding. Elasticsearch or OpenSearch fits when BM25 is already central and vectors are the incremental step. Redis fits when in-memory hybrid search matches corpus size and RAM budget.

If BM25 is already central, Weaviate still belongs in the shortlist when you are willing to run a dedicated vector database, and Elasticsearch or OpenSearch belong when extending the lexical cluster is the organisational path. Those are different decisions, not interchangeable brand names.

How should you evaluate the shortlist on your own data?

You should evaluate the shortlist on your own data with a labelled query set, separate retrieval scoring, and filtered plus exact-token slices that match production. ZenML’s comparison framing and IBM’s RAG substrate framing both collapse if you only read generated answers. Stage separation is mandatory.

Hold embedding model and chunking constant while swapping stores, then hold the store constant while swapping chunking. Otherwise you cannot tell which layer moved the metric. Record as-of dates for every vendor claim you relied on during the bake-off.

Refuse to cite a latency number that you did not measure on your hardware or that a vendor did not publish on a named page captured for this rebuild. Not published is an allowed cell. Invented milliseconds are not.

What should you verify before you commit?

Before you commit, verify delete propagation, filter behaviour under load, hybrid or lexical paths for exact tokens, backup and restore ownership, and client version pins against official docs. Verify cost meters or hardware plans with finance using published figures only.

Also verify who wakes up when recall drops. If that name is blank, managed paths beat heroic self-hosting. If residency forbids managed paths, staff the self-host path before traffic arrives.

Then commit to one store for a measured period, keep the survey and profiles bookmarked for the next architecture review, and do not churn stores weekly based on social media anecdotes.

Who owns upgrades, restores and recall monitoring?

Upgrades, restores and recall monitoring decide whether a store survives production, because a good neighbour on a laptop does not prove deletes propagate, filters still hold under load, or an embedding model swap is followed by a full re-index. Those are ownership questions as much as product questions, and they belong in the runbook before traffic arrives.

Name the person who upgrades the version, the person who restores from backup, and the dashboard that shows retrieval recall separately from generation scores. If those names are blank, prefer a managed path or delay production claims until staffing exists. Capacity planning should use your real dimension width, object count and filter cardinality rather than a tutorial corpus.

How should you evaluate retrieval after you pick the store?

You should evaluate retrieval after you pick the store by keeping a labelled query set, scoring neighbours before generation, and re-running after every schema, filter or embedding change. Generation fluency will hide wrong chunks if you only read final answers. Stage separation is how you know whether to change the store, the chunker or the prompt.

Include filtered queries in the set if production uses filters. Unfiltered recall can look healthy while tenant or source filters destroy candidates. Include exact-token queries if your users ask for error codes and SKUs; dense-only stores will fail those unless hybrid or lexical paths exist. Document the as-of date of every vendor claim you relied on, and treat unpublished numbers as not published.

How does this store sit beside Weaviate and neighbouring profiles?

This store sits beside Weaviate and neighbouring profiles as one shape in an ordered field, not as a universal winner. Weaviate remains the batteries-included open-source baseline with modules when that control model matters. Neighbouring managed and self-host profiles cover serverless, filtering-first and in-database extension paths without requiring this profile to invent a latency leaderboard.

Use the category hub at vector databases for definitions and the survey at which vector database should you use for RAG for scored fit across stores. Keep internal links in prose, keep sources named rather than linked externally, and keep every volatile claim dated in the same sentence that carries it.

What else should you confirm before you scale beside Weaviate and neighbouring profiles?

This store sits beside Weaviate and neighbouring profiles as one shape in an ordered field, not as a universal winner. Weaviate remains the batteries-included open-source baseline with modules when that control model matters. Neighbouring managed and self-host profiles cover serverless, filtering-first and in-database extension paths without requiring this profile to invent a latency leaderboard.

Use the category hub at vector databases for definitions and the survey at which vector database should you use for RAG for scored fit across stores. Keep internal links in prose, keep sources named rather than linked externally, and keep every volatile claim dated in the same sentence that carries it.

The right vector database for RAG is the one that matches your filtered hybrid needs, operations staffing and cost model, scored here with an editorial fit rubric against Weaviate, Qdrant, Pinecone, Milvus, Chroma, pgvector, FAISS, Elasticsearch or OpenSearch, and Redis using September 2026 docs. The table sorts by score and is not a latency benchmark. Keep profile pages for honest limits, measure retrieval on your queries, and refuse unpublished numbers rather than inventing them.