IVF, Flat and the Recall You Trade for Speed

IVF (inverted file) indexes partition embedding space into clusters and search only a subset of those clusters at query time, while a flat index compares the query to every vector and stays exact until the collection outgrows the latency budget. Weaviate’s flat and dynamic index docs sit beside Pinecone’s nearest-neighbour index series, Milvus’s IVF_FLAT reference, Oracle’s inverted-file-flat notes, TigerData’s vector-index basics and Teradata’s ANN primers (extracted September 2026) on that partition-versus-scan choice as the core recall-latency-memory trade. Weaviate’s vector-index concepts page (as of September 2026) documents flat as the lightweight list for small collections and dynamic as the path that starts flat then switches to HNSW past a scale threshold; partitioned IVF depth in other stacks sits beside those options on structures, trade-offs and limits. The useful question is not which name sounds modern, but which structure matches your write pattern, memory ceiling and measured recall.

What is an IVF index in vector search?

An IVF index is an inverted-file structure over vector clusters: the builder trains a set of centroids, assigns each stored vector to its nearest centroid, and at query time probes only the nprobe closest lists instead of the whole corpus. Weaviate’s flat-index guidance for small shards sits beside Pinecone’s inverted-file chapter and Milvus’s IVF_FLAT docs, which describe that coarse-quantization-plus-list-scan pattern as the way partitioned search stays sublinear while remaining simpler than a multi-layer proximity graph. Oracle’s inverted file flat documentation frames the same idea for AI Vector Search: inverted lists over partitions, then a finer scan inside the selected lists.

Weaviate teams on flat or HNSW will not see these names; on FAISS and Milvus stacks the partition count and probe count (nlist and nprobe) are the two dials that dominate the curve. Raising nlist makes lists smaller and can cut work per probe when centroids fit the data; raising nprobe visits more lists and usually raises recall at the cost of latency. Those names are FAISS-family conventions; product UIs may hide them behind accuracy presets, but the mechanism is still “how many clusters” and “how many clusters to visit”. IVF is approximate because a true neighbour sitting in an unprobed list is invisible to that query.

What is a flat index, and when is exhaustive search enough?

A flat index stores every vector in one list and searches through all of them to find nearest neighbours. Weaviate’s concepts page (as of September 2026) states that flat is extremely memory-efficient relative to graph indexes, that search time grows linearly with the number of vectors, and that flat is a good choice for small collections such as multi-tenancy use cases where each tenant holds an isolated dataset. The same page notes that flat does not scale like HNSW’s logarithmic query complexity.

Exhaustive search is enough when measured p95 retrieve latency on your queries already meets the budget at the current object count. You get exact neighbours, skip graph construction, and avoid underbuilt edges. Weaviate’s dynamic index (default threshold 10,000 objects with async indexing) automates the later switch to HNSW when the shard grows. Qdrant’s essentials course documents a parallel full-scan threshold pattern for small collections on its stack. Prefer flat while the set is small; do not keep scanning millions of vectors because “exact” sounds safer than a measured ANN configuration.

How does IVF build its clusters?

IVF build starts by learning centroids, usually with k-means or a related clustering pass over a training sample of vectors, then assigning every corpus vector to its nearest centroid so each cluster becomes an inverted list. Weaviate’s index-type menu (flat, HNSW, dynamic) is the contrasting product path; Pinecone’s FAISS notebook path and Milvus’s build-parameter sections separate that training step from the later add step: bad centroids from a tiny or biased training sample produce lists that do not match the live distribution. VeloDB and The Data Quarry write-ups in the live teardown set likewise treat centroid quality as a first-order recall lever rather than as an implementation detail.

After assignment, optional product quantization can compress vectors inside each list (IVFPQ in FAISS vocabulary), but compression maths belong on quantization; here the rule is only that IVF and PQ often travel together when memory is tight. Rebuild cost matters in production: when inserts shift the distribution, old centroids mis-route queries. That is the write-pattern difference versus HNSW’s incremental graph updates on CRUD-capable stores such as Weaviate. Plan a rebuild window before you treat IVF as “set and forget”.

How does IVF search with nprobe?

IVF search embeds the query, finds the nearest centroids, opens the corresponding inverted lists, and scores vectors inside those lists to return top-k. nprobe is the number of lists visited. Weaviate measures recall under the index you configured; Milvus’s index-specific search params and Pinecone’s balancing chapter show the familiar curve: low nprobe is fast and miss-prone; high nprobe approaches flat cost as you open more of the corpus. Teradata and TigerData primers make the same qualitative point without claiming a universal Recall@k.

Because unprobed lists are invisible, filtered search interacts with IVF differently than with HNSW. Coarse filters at the centroid or list level can stay stable when graph traversal under hard predicates collapses; that interaction is why some production teams prefer partitioned indexes under heavy metadata load. Filter mechanics themselves live on metadata filtering; the index-level point is that partition shape changes which failure you see first when predicates are selective.

What recall, latency and memory does IVF trade for speed?

IVF trades the chance of missing neighbours in unprobed lists for fewer distance computations and usually lower RAM than a dense HNSW graph. BigData Boutique’s Syn-Hershko write-up (29 May 2026), already used site-wide for indexing comparisons, reports IVFFlat memory near about 1.1 times raw vectors versus HNSW overhead roughly 2 to 5 times, and notes IVFFlat recall drift as data changes while HNSW absorbs inserts without a full rebuild. Those figures are vendor-reported structural claims; they are not an SLA for your corpus.

Milvus IVF-versus-HNSW guidance in related indexing teardowns reports mid-90s percent recall bands for IVF without compression in some of its own benchmarks, with HNSW usually sitting higher, and notes filtered-search behaviour that can favour IVF’s coarse path. Treat every published percentage as a starting hypothesis. Set your own recall@k and p95 retrieve_ms targets, sweep nprobe first (query-time), then revisit nlist and rebuild cadence if the distribution has moved. When memory still fails after a sane IVF setup, open quantization before you abandon partitioned search.

When should you choose IVF instead of HNSW?

Choose IVF when memory is tight relative to an in-memory graph, when rebuild windows are acceptable, or when heavy metadata filters make cluster-level coarse filtering more stable than graph traversal under sparse legal sets. Choose HNSW when RAM allows, documents keep arriving, and you want incremental inserts without centroid rebuilds. Weaviate’s default for most collections (as of September 2026) is HNSW; flat covers small shards; dynamic bridges flat to HNSW at a threshold. IVF appears beside Weaviate’s flat and HNSW options in the wider ecosystem, including FAISS, Milvus and related stacks as the partitioned alternative those products document beside HNSW.

Prefer flat while exhaustive scan meets latency. Prefer HNSW depth on HNSW explained when graph parameters and memory models are the real question. Prefer quantization when the vectors themselves are the RAM problem. Prefer updates and deletes documentation when stale centroids are really a mutate-path bug. Naming Weaviate, Pinecone, Qdrant and Milvus does not pick the index: the same product family can expose more than one structure, and the wrong structure inside a strong store still returns the wrong neighbours.

Why do IVF indexes need rebuild discipline?

IVF indexes need rebuild discipline because centroids are a snapshot of the training distribution. Continuous inserts, deletes and domain shift move mass in embedding space; queries then probe the wrong lists even when nprobe looks generous. Weaviate’s rebuild story for graphs is async cleanup; Pinecone’s and Milvus’s operational notes around IVF build parameters imply that training quality and list balance are ongoing concerns, not one-time setup. OpenSearch forum threads and practitioner write-ups in adjacent update SERPs show the same symptom from the other side: “we added documents and recall fell” often means the partition map aged out.

HNSW is not immune to quality loss, but its failure modes differ: underbuilt edges, tombstones before cleanup, and filter collapse on sparse graphs. Weaviate’s async cleanup after deletes and updates (concepts page, September 2026) is the graph analogue of “the structure still thinks the old object is connected”. For IVF, the analogue is stale centroids. Change detection that decides when to rebuild or re-embed lives on incremental indexing; the mutate APIs live on updating and deleting documents.

What do nlist and nprobe control in practice?

Beside Weaviate’s flat and HNSW defaults, FAISS-family and Milvus vocabulary use nlist is how many clusters the builder creates and nprobe is how many of those lists a query opens. Raising nlist tends to shrink each list when the training sample matches the corpus, which can cut work per probe; raising it too far on a small corpus creates empty or tiny lists that waste probes. Raising nprobe is the usual first latency-versus-recall sweep because it is often a query-time setting. Weaviate’s own serving default for most collections remains HNSW rather than IVF, so these knobs appear when you are on a FAISS, Milvus or similar partitioned path, or when a managed service exposes equivalent accuracy presets over clusters.

Supabase and SkyZH teardown pages show practitioners discovering that copying a blog’s nlist without measuring list balance produces uneven latency. Balance the training sample, watch list size histograms if the engine exposes them, and refuse to treat a preset labelled “high accuracy” as a measured Recall@k on your queries. When the product hides nprobe behind a percentage accuracy target, still verify with an exact-neighbour sample on a held-out query set.

Filtered queries change the effective curve. If metadata predicates remove most of a list, a nominal nprobe may still under-deliver legal neighbours. That is the moment to compare partitioned search against filter-aware HNSW behaviour rather than to keep raising probes blindly. Weaviate’s filter documentation and the filtering leaf own the predicate side; here the rule is to measure filtered recall whenever predicates are part of production traffic.

How does IVF relate to LSH and other ANN families?

Locality-sensitive hashing and other ANN families appear in Pinecone’s broader index survey beside IVF and HNSW. They hash nearby vectors into shared buckets so search can probe buckets instead of clusters or graphs. LSH stays outside the production leaf set used here: depth concentrates on flat, HNSW, IVF, quantization, filtering, multi-tenancy and updates. If a measured experiment shows those six cannot meet a latency or memory bar, document the experiment; do not silently redefine indexing as hashing.

Tree-based indexes appear in Weaviate’s concepts overview as another historical family that can be memory-efficient for some low-dimensional cases and costly to update when trees need rebuilds. Graph indexes such as HNSW are the Weaviate default. Cluster-based and flat options complete the menu. IVF in FAISS-shaped stacks is the practical cluster path most RAG teams meet after flat and HNSW.

How do FAISS flat and IVF indexes show up in RAG stacks?

FAISS popularised the flat, IVF and IVFPQ vocabulary many RAG engineers still use even when the serving database is not FAISS. Pinecone’s index series walks flat accuracy, IVF implementation and HNSW in one tutorial arc; VeloDB and The Data Quarry pieces in the live teardown set compare those structures for analytical and serving workloads. Weaviate’s product path emphasises HNSW, flat and dynamic instead of exposing classic FAISS IVF as the default, which is why this leaf teaches IVF mechanics for the stacks that ship them while keeping Weaviate’s flat guidance as the small-shard exact path.

When a team says “we use IVF in production,” ask which product API they mean, what nlist and nprobe equivalents are set, whether PQ is attached, and when centroids were last retrained. Without those four answers, the name IVF is only a badge. Pair the answers with a measured recall@k on filtered and unfiltered traffic before you change embedding models.

How do you decide the flat-to-ANN cutover point?

Decide the flat-to-ANN cutover with a latency budget and a growth forecast, not with a folklore vector count. Weaviate’s dynamic index default threshold of 10,000 objects (with async indexing) is one published cutover. Qdrant’s example full_scan_threshold of 10,000 is another. Your cutover may be lower if dimensions are wide and distance compute is expensive, or higher if queries are rare and exactness is mandatory.

Run a soak: measure p95 retrieve time on flat as object count climbs on a realistic query mix, including filters. Cut over when flat breaches the budget with headroom for growth, then re-measure ANN recall against a flat sample so you know what approximation costs. Dynamic indexes automate the swap; they do not automate the recall check.

Exact search remains the yardstick for every approximate configuration. Oracle’s inverted-file documentation and Pinecone’s flat-index chapter both keep exhaustive search in the story so practitioners can see what IVF gives up. Weaviate’s flat index is the productised form of that yardstick for small collections: every vector stays searchable without cluster assignment. When an IVF deployment cannot beat flat on a sample of queries at the same k, the partition map is wrong, the probe count is too low, or the problem was never index shape.

Hybrid retrieval does not replace IVF. Lexical signals still need their own inverted index for terms; vector IVF only partitions embeddings. Teams sometimes conflate “inverted file” for BM25 with “inverted file” for vectors. Keep the lexical path on BM25 and hybrid retrieval, and keep this leaf focused on vector partitions.

Memory accounting should include centroid storage, inverted-list ids and any PQ codes attached to list entries. IVFFlat-style indexes stay closer to raw vector size than dense graphs in the BigData Boutique comparison already cited, which is why teams under RAM pressure evaluate partitions before they abandon approximate search. That advantage disappears if you set nprobe so high that every query touches most lists: you have reinvented flat search with extra bookkeeping.

Documentation quality varies by engine. Prefer the primary product docs for parameter defaults, then use survey posts for intuition. Teradata and TigerData primers in the live teardown set are intuition sources; Milvus and Oracle pages are closer to operator manuals for their IVF flavours. Weaviate remains the default example for flat and HNSW behaviour on this site when the question is which structure a CRUD vector database starts from.

When centroids and embeddings disagree because you changed embedding models without rebuilding lists, IVF fails loudly as recall collapse. Re-embed and retrain partitions together. The same-model rule from the embeddings hub applies inside every inverted list.

SkyZH’s from-scratch vector database notes in the teardown set are a useful reminder that inverted lists are a data structure you can implement badly. Production engines spend years on list layout, SIMD distance and concurrency; your job is to set probes and rebuild cadence, not to reimplement IVF.

IVF earns its place when partitions cut work enough to hit latency under a memory budget you can actually buy, and when you can schedule centroid refresh. Flat earns its place while the shard is small enough for an exact scan. HNSW earns the default when RAM and continuous writes dominate. Measure recall under your filters, sweep probe count before you blame the embedder, and rebuild when the distribution has moved rather than when a blog promises a magic nlist.

More in Indexing for RAG: Structures, Trade-offs and Limits Toggle section list