Vertex AI Vector Search for RAG

Vertex AI Vector Search is Google Cloud’s managed approximate nearest-neighbour service, built on Google Research’s ScaNN algorithm and formerly known as Matching Engine. For RAG it scales similarity search; you still own chunking, embeddings, and orchestration unless you move up the GCP stack to RAG Engine or Vertex AI Search. The trade-offs are Google Cloud lock-in and always-on index cost.

What is Vertex AI Vector Search?

Vertex AI Vector Search is a managed vector search engine on Google Cloud that finds similar embeddings at scale using the ScaNN algorithm. Google’s product docs (captured July 2026) frame it as the same research lineage that powers large Google products, offered as an enterprise ANN service for semantic search, recommendations, and generative-AI retrieval. The older name Vertex AI Matching Engine still appears in blogs and harvested queries — it is the prior product name for this vector search line.

That makes Vector Search a specialist retrieval store, not a general database. If you only need the category background, start at vector databases. The rest of this profile stays on what Vertex gives a RAG team, what it leaves for you to build, and what the GCP-only boundary costs.

How does Vertex AI Vector Search fit into a RAG stack?

Vertex AI Vector Search is the retrieval component, not a turnkey RAG product. Saurabh Pandey’s Google Cloud Community guide on the GCP RAG spectrum puts three products on one ladder: Vector Search at low abstraction (maximum control), Vertex AI RAG Engine in the middle (managed orchestration), and Vertex AI Search at high abstraction (turnkey enterprise search). With Vector Search alone, you generate embeddings, build and deploy an index, query top-k neighbours, and wire generation yourself.

Google’s docs also document hybrid search over dense and sparse embeddings, and streaming index updates for fresher corpora — the spectrum guide notes that a streaming index is a prerequisite when Vector Search backs Vertex AI RAG Engine. The fusion maths for combining keyword and semantic scores belong at hybrid search; embedding choice belongs at embeddings; end-to-end wiring belongs at how to build a RAG pipeline.

Do not confuse the names. Vertex AI Search is the high-abstraction search application. Vertex AI Vector Search is the ANN index service underneath many custom RAG designs. Mixing them up is the most common GCP RAG mistake the spectrum article exists to prevent.

What is Agent Retrieval (formerly Vector Search 2.0)?

Agent Retrieval — formerly marketed as Vector Search 2.0 — is Google Cloud’s next managed path: instead of treating the ANN index as the only primary resource, you work with Collections of Data Objects, a unified storage and retrieval surface that removes the usual “vector index plus separate feature store” stitch. Google’s Agent Retrieval overview (last updated 23 July 2026 in the capture) lists auto-populated embeddings or bring-your-own embeddings, rich filtering, and Collection indexes for ANN scale, plus usage-based and resource-based pricing models.

Kaz Sato’s Google Cloud Community introduction to Vector Search 2.0 describes the operational ramp: start with exact k-nearest neighbours for development, then add an ANN/ScaNN index for production, with the article claiming sub-10 ms latency even at billion-vector marketing scale. Treat that latency figure as a vendor-community claim, not an independent benchmark on your corpus. Supported regions on the Agent Retrieval docs page in the July 2026 capture include us-central1, us-east4, us-west1, several Europe and Asia regions — verify the live region list before you design multi-region failover. Filter expression design more broadly sits at metadata filtering.

What are Vertex AI Vector Search’s capabilities and limits, side by side?

Vertex AI Vector Search is easiest to judge when each strength is paired with the ceiling that rides along.

The cost row is where teams get surprised after the first prototype, so pricing gets its own section.

How much does Vertex AI Vector Search cost?

Vertex AI Vector Search pricing is driven by the virtual machines that host deployed indexes plus charges to build and update indexes. Google’s Vector Search docs (July 2026) say even a minimal setup can accommodate moderate throughput for under $100 per month — that is a vendor estimate, so run the Gemini Enterprise Agent Platform Vector Search calculator before you budget.

The harder engineering fact is utilisation shape. The GCP RAG spectrum guide states that classic Vector Search keeps a provisioned index endpoint active and accruing cost even when query traffic is sporadic. That always-on model is attractive for steady high-QPS retrieval and painful for bursty demos. Agent Retrieval’s docs describe adaptable usage-based and resource-based pricing as the newer path for smaller or tuned workloads — still verify live meters rather than treating “under $100” as a fixed SKU.

This profile does not invent per-node hourly rates for Vector Search from adjacent Vertex AI training tables. If a number is not clearly labelled Vector Search / Agent Retrieval on the pricing page you are reading, do not copy it into a RAG bill of materials. Cost belongs in the broader vector-database decision once you know your QPS and freshness requirements.

Vertex AI Vector Search vs Weaviate: which trade-off matters for RAG?

The choice is usually Google Cloud scale versus portable control. Vertex AI Vector Search optimises for managed ScaNN ANN inside GCP. Weaviate optimises for an open-source vector database you can self-host, with native hybrid search and modules outside a single cloud control plane.

Pick Vertex when you already run on Google Cloud, need that managed ANN path, and accept always-on index economics. Pick Weaviate when portability, local development parity, or avoiding GCP lock-in is the bottleneck. The spectrum guide even notes that Vertex AI RAG Engine can use Weaviate as a vector backend — evidence that Google’s own mid-layer treats open vector stores as interchangeable components, not as heresy. The scored multi-database verdict still belongs at which vector database should you use for RAG.