Vertex AI Vector Search for RAG

Vertex AI Vector Search is Google Cloud’s managed approximate nearest-neighbour service, built on Google Research’s ScaNN algorithm and formerly known as Matching Engine. For RAG it scales similarity search; you still own chunking, embeddings, and orchestration unless you move up the GCP stack to RAG Engine or Vertex AI Search. The trade-offs are Google Cloud lock-in and always-on index cost for classic provisioned endpoints.

What is Vertex AI Vector Search?

Vertex AI Vector Search is a managed vector search engine on Google Cloud that finds similar embeddings at scale using the ScaNN algorithm. Saurabh Pandey’s Google Cloud Community spectrum guide (August 2025; live capture September 2026) frames it as the same research lineage that powers large Google products, offered as an enterprise ANN service for semantic search, recommendations, and generative-AI retrieval. The older name Vertex AI Matching Engine still appears in blogs and harvested queries; it is the prior product name for this vector search line.

That makes Vector Search a specialist retrieval store, not a general database and not a turnkey RAG product. The spectrum guide is explicit: when you use Vector Search directly, the developer owns data ingestion, document parsing and chunking, embedding generation, and the final generation step with an LLM. Category background for the wider field sits at vector databases. The rest of this profile stays on what Vertex gives a RAG team, what it leaves for you to build, and what the GCP-only boundary costs.

Pandey also lists primary DIY use cases that are not limited to chat RAG: custom recommendation engines, fraud or anomaly detection by similarity, ad-serving platforms, and bespoke classification chatbots. Those examples matter because they show Vector Search as a similarity primitive, not only as an LLM context store. If your workload is pure ANN without generation, you still evaluate the same index, endpoint, and always-on cost shape.

Google’s architecture guide for generative AI RAG with Agent Platform and Vector Search (September 2026 capture) situates the same product inside a larger ingestion-and-serving subsystem rather than as a chatbot kit. Personalized recommendations, clinical assistance, and legal research appear there as example shapes, not as published accuracy scores for your corpus.

How does Vertex AI Vector Search fit into a RAG stack?

Vertex AI Vector Search is the retrieval component, not a complete RAG application. Pandey’s spectrum guide puts three products on one ladder: Vector Search at low abstraction (maximum control), Vertex AI RAG Engine in the middle (managed orchestration), and Vertex AI Search at high abstraction (turnkey enterprise search). With Vector Search alone, you generate embeddings, build and deploy an index, query top-k neighbours, and wire generation yourself.

The same guide documents hybrid search over dense and sparse embeddings, plus multimodal embeddings for images, audio, and video when you supply those vectors. Index update mode is a first-class choice: BATCH_UPDATE for periodic refreshes, and STREAM_UPDATE for near-real-time upserts. A streaming index is described as a prerequisite when Vector Search backs Vertex AI RAG Engine. Fusion maths for combining keyword and semantic scores belong at hybrid search; embedding choice belongs at embeddings; end-to-end wiring belongs at how to build a RAG pipeline.

Do not confuse the names. Vertex AI Search is the high-abstraction search application. Vertex AI Vector Search is the ANN index service underneath many custom RAG designs. Mixing them up is the most common GCP RAG mistake the spectrum article exists to prevent. Google’s RAG Engine docs for using Agent Platform Vector Search (September 2026 capture) show the operational path: create an index, create an index endpoint, deploy the index, then point a RAG corpus at that endpoint. Arjun Prabhulal’s public Gemini-embeddings-plus-Vector-Search repository and OneUptime’s February 2026 RAG Engine walkthrough both treat Vector Search as the store behind a DIY or managed corpus, not as the whole product.

What is Vector Search 2.0 on Google Cloud?

Vector Search 2.0 is Google Cloud’s next managed path described in the developers.google.com codelab “Introduction to Vector Search 2.0” (live capture September 2026). Instead of treating the classic ANN index as the only primary resource, the codelab frames Collections of data objects as a unified storage and retrieval surface that reduces the usual vector-index-plus-separate-feature-store stitch.

The codelab lists auto-populated embeddings or bring-your-own embeddings, rich filtering, and ANN indexes for production scale. It also claims approximate nearest-neighbour search with “sub-10ms latency even with billions of vectors” and roughly 99% accuracy trade-offs versus exact search. Treat those latency and accuracy figures as vendor-codelab claims, not an independent benchmark on your corpus. Exact k-nearest neighbours for development, then ANN/ScaNN for production, is the ramp the same codelab describes.

Task-type embeddings and hybrid search demos in that codelab matter for RAG because they show Google steering developers toward query and document embedding modes that match the retrieval job. Filter expression design more broadly sits at metadata filtering. If your team still runs classic Matching Engine style indexes, keep the older create-deploy-query lifecycle in mind until you intentionally migrate to the Collections path.

What are Vertex AI Vector Search capabilities and limits side by side?

Vertex AI Vector Search is easiest to judge when each strength is paired with the ceiling that rides along. The rows below come from the spectrum guide, the architecture guide for RAG with Vector Search, and the Vector Search 2.0 codelab (all September 2026 captures).

The cost row is where teams get surprised after the first prototype, so pricing gets its own section. Index create-to-deploy time can also take up to an hour according to the spectrum guide, which matters for CI/CD agility even when query latency is fine. Elasticsearch-plus-Vertex tutorials that ranked beside Vector Search pages show another pattern: using Vertex models with a different vector store. That is a valid stack choice, but it is not the subject product for this profile.

How much does Vertex AI Vector Search cost?

Vertex AI Vector Search pricing is driven by the virtual machines that host deployed indexes, plus charges shaped by index size and query load. Google’s architecture guide for generative AI RAG with Vector Search (September 2026 capture) states that billing depends on the size of your index, queries per second, and the number and machine type of the nodes used for the index endpoint. For high-QPS workloads it suggests batching queries to reduce cost, and it documents autoscaling so node count can fall toward a configured minimum when demand is low.

The harder engineering fact is utilisation shape. Pandey’s spectrum guide states that classic Vector Search keeps a provisioned index endpoint active and accruing cost even when query traffic is sporadic. That always-on model is attractive for steady high-QPS retrieval and painful for bursty demos. The same guide contrasts that shape with usage-based options higher on the spectrum (Vertex AI Search) or serverless BigQuery vector search for low-traffic hobbies. A specific monthly dollar floor for a minimal Vector Search setup is not published in the organic pages captured for this rebuild; do not invent one from adjacent Vertex AI training tables.

A Google Developer forum thread in the live set (January to February 2025 posts; September 2026 capture) shows practitioners still asking how many vectors an index can hold and how to combine Vertex AI Search grounding with Vector Search cost lines. Treat that as evidence of pricing opacity, not as a published capacity number. Use the Gemini Enterprise Agent Platform Vector Search pricing examples linked from the architecture guide before you budget. Cost belongs in the broader vector-database decision once you know your QPS and freshness requirements.

How does Vertex AI Vector Search compare with Weaviate for RAG?

The choice is usually Google Cloud scale versus portable control. Vertex AI Vector Search optimises for managed ScaNN ANN inside GCP. Weaviate optimises for an open-source vector database you can self-host, with native hybrid search and modules outside a single cloud control plane. When RAG Engine backends are named together, Weaviate leads the peer list beside Pinecone in Pandey’s spectrum guide as supported third-party options.

Pick Vertex when you already run on Google Cloud, need that managed ANN path, and accept always-on index economics for classic endpoints. Pick Weaviate when portability, local development parity, or avoiding GCP lock-in is the bottleneck. Teams already deep in BigQuery may also weigh BigQuery’s serverless vector search path that Pandey names for low-traffic hobbies; that option sits outside this Vector Search profile, but it explains why cost, not only recall, pushes people off provisioned ANN endpoints.

Operationally, classic Vector Search asks for index creation, endpoint creation, and deploy before the first production query lands. RAG Engine collapses much of that into a corpus object when you accept its orchestration, while still allowing Weaviate or Pinecone as the store under that corpus. That is why the comparison is not only “Vertex versus Weaviate as products” but also “which abstraction layer you want to live on inside GCP.” The spectrum guide even notes that Vertex AI RAG Engine can use Weaviate as a vector backend, evidence that Google’s own mid-layer treats open vector stores as interchangeable components. Pinecone appears in the same backend list for teams that want another managed option without leaving the RAG Engine corpus abstraction.

Neither product wins a scored bake-off in a single-vendor profile. The scored multi-database verdict still belongs at which vector database should you use for RAG, where latency, hybrid quality, and total cost can be compared under one rubric rather than inside a single-vendor profile.

For RAG teams already committed to Google Cloud, the practical sequence is usually: prove retrieval quality on a streaming or batch index with your own embeddings, measure endpoint utilisation under realistic QPS, then decide whether RAG Engine’s corpus abstraction is worth the loss of DIY control. Skip inventing monthly dollars from marketing pages that do not label Vector Search line items; verify the live calculator linked from the architecture guide instead.

Vertex AI Vector Search is the low-abstraction ANN service on Google Cloud: ScaNN-backed retrieval you wire into your own chunking, embedding, and generation stack, with Vector Search 2.0 as the Collections-oriented path and RAG Engine or Vertex AI Search as higher-abstraction siblings. Budget for provisioned endpoints and verify live node pricing; choose Weaviate when you need an open, portable vector database instead of a GCP-native index.