IVF, Flat and the Recall You Trade for Speed
Exhaustive versus partitioned search, with the recall-latency curve rather than a claim about it.
IVF and flat indexes are two approaches to nearest-neighbour search that trade work per query: IVF partitions vectors into centroid buckets and searches only the closest n_probe clusters, while FLAT compares the query to every vector for exact results.
How does IVF build its index?
IVF builds its index by first clustering vectors into n_list centroids (often via k-means) and then assigning each vector to its nearest centroid to form inverted lists (posting lists) for later search. Oracle describes IVF as a storage-based partitioning index, in contrast to in-memory graph indexes like HNSW.
A typical build has two phases:
- Cluster into n_list centroids. The centroid table defines the coarsest partition of the vector space.
- Assign vectors to centroids. Each vector becomes a member of one inverted list (the list for its nearest centroid).
How does IVF search for nearest neighbors?
At query time IVF narrows the search area by selecting which inverted lists to scan: it computes distances between the query vector and all centroids, sorts centroids by distance, then searches only the closest n_probe clusters. The recall outcome depends on whether the correct centroid clusters are among those probed lists.
Milvus describes the core query steps as:
- Compute query-to-centroid distances.
- Select the nearest n_probe clusters.
- Search within those clusters. You compare the query to vectors only inside the selected inverted lists.
What do n_list and n_probe control in IVF?
In IVF, n_list controls how many centroids (and inverted lists) exist, while n_probe controls how many of those lists get searched per query. Zvec suggests a rule of thumb to start with n_list ≈ √N (N = vectors in the shard), and Milvus adds a practical example: for a million vectors, try n_list = 1,000. Milvus also notes that you can often test n_probe dynamically (for example values from 1 to 16) to find an acceptable recall/latency sweet spot.
How do IVF and flat indexes trade recall for speed?
FLAT provides exact nearest neighbours by brute-force comparing each query to every vector, and Milvus describes this as guaranteeing 100% recall. The downside is full-scan latency at scale. IVF reduces work by scanning only a subset of vectors inside the selected centroid clusters; with too-small n_probe, IVF can miss some true top-k neighbours.
That’s why recall in IVF is best understood as a curve you tune, not a universal constant. For example, Milvus’ IVF variant comparison describes that with a 64:1 compression ratio, IVF_PQ recall is typically around 70%, and it can reach 90%+ by lowering compression (using milder compression settings).
How much memory does IVF use compared to HNSW and flat?
Oracle describes IVF as a storage-based index rather than an in-memory index, so the “actual index” itself is not the same kind of RAM graph footprint as HNSW. Oracle also notes that IVF still needs memory for centroid/index creation and DML operations if the database defines a vector pool, but the index is not constrained the way an in-memory graph is.
For comparison:
- HNSW stores a proximity graph in memory, which can be heavy at large scale.
- FLAT needs no training and no advanced preprocessing, but it still performs a full scan per query.
- IVF focuses on partitioning and can be more memory-friendly for large datasets.
When should you use IVF instead of HNSW or flat?
Use IVF when you want partition-based ANN search for very large collections and you cannot afford the in-memory graph footprint of HNSW, but you also need better query performance than exact FLAT scanning. Oracle describes IVF as working for very large datasets and still providing excellent performance versus exhaustive similarity search, even though it won’t match an equivalent HNSW index’s raw speed.
For RAG specifically, decide using measured retrieval quality on your own query set: run FLAT as an exact baseline on a sample, then sweep n_probe and measure when the top-k results stop containing the gold chunks your answers require.
When is IVF the wrong index?
IVF is the wrong index when exact nearest neighbours are non-negotiable (flat provides exact results via brute-force) or when n_probe is tuned too low for your recall needs, causing the query to miss the correct centroid clusters. IVF is also a poor fit if your workload demands rapid index maintenance without any tolerance for rebuilding centroid partitions.
If your issue looks like “answers cite the wrong context,” validate whether it is a partition/probe miss (try higher n_probe) before treating the problem as a model or chunking failure; broader symptom diagnosis lives on leakage and other /failures pages.
What is IVF in vector search?
IVF (Inverted File) in vector search partitions the embedding space by clustering vectors into n_list centroids. At query time, IVF computes distances to centroids and searches only the closest n_probe clusters (inverted lists), instead of scanning every vector. The trade-off is that small n_probe can reduce recall.
What does n_list mean?
n_list is the number of centroids (and inverted lists) created during IVF indexing. Zvec’s rule of thumb starts with n_list ≈ √N (N = vectors in the shard), and Milvus gives a concrete example: for 1 million vectors, try n_list = 1,000.
What does n_probe do?
n_probe is the number of clusters IVF searches at query time. Larger n_probe typically improves recall but increases latency, while smaller n_probe speeds up queries but may miss some true nearest neighbours. Milvus notes that you can experiment dynamically with values like 1 to 16 to find the recall/latency sweet spot.
Is FLAT always exact?
FLAT is exact in the sense that it brute-forces the query against every vector, and Milvus describes this as guaranteeing 100% recall. The downside is that full scans make FLAT slow as datasets grow.
Is IVF always worse than HNSW?
No. IVF and HNSW target different trade-offs. Oracle describes IVF as storage-based (not an in-memory graph) and better suited for very large datasets when you cannot afford HNSW’s in-memory footprint, while also noting that IVF won’t match an equivalent HNSW’s raw speed. In RAG, you still choose by measured recall@k and retrieve latency on your own query set.