Keeping a RAG Index Current: Incremental Updates

Incremental indexing updates a RAG vector index by detecting which documents are new, changed, or deleted, then embedding only those, instead of re-chunking and re-embedding the whole corpus on every sync. The first load still belongs under document ingestion. After that load, change detection, three write paths, cost versus full rebuild, event versus batch cadence, and the cases where a full reindex is still mandatory decide whether answers stay fresh. Without a delete path, removed policies keep retrieving. Without change detection, rewritten SOPs keep answering from the old embedding. Those failures look like generation bugs until the index is inspected.

How does incremental indexing detect changed documents?

Incremental indexing detects changes by comparing each source document to a stored fingerprint, usually a content hash, and classifying it as new, changed, unchanged, or deleted before any embedding call runs. TypeGraph (2026) states the four-way split explicitly: no prior hash means new; a differing hash means changed; a matching hash means skip; a hash present in the store but absent from the source means delete. The hash comparison itself is cheap. TypeGraph reports sub-second comparison even at about a million documents. The expensive work, chunk, embed, write, runs only on the changed set.

Three detection methods show up across production guides. They are not interchangeable. Each catches a different class of change and misses another. A defining lead belongs before any table of methods, because the table is furniture for a decision already stated in prose.

In practice teams combine them: a cursor or webhook nominates candidates, a content hash decides whether to re-embed, and an end-of-run inventory reconcile catches deletes the event stream missed. Particula’s November 2025 guide and SimpleVector’s change-tracking patterns both treat that hybrid as the default production shape rather than as an exotic option.

Detection alone does not update the index. The next decision is which write path runs for each classified document: insert, replace, or prune.

How do you add, update, and delete documents without a full reindex?

Incremental sync has three write paths: insert new vectors, replace changed documents, and prune deleted ones. All three need a metadata registry keyed by source_id, with document ID, content hash, and ingestion time. Particula (Sebastian Mondragon, November 2025) treats those three operations as the whole update surface. Most “rebuild everything” pipelines exist because only the first path was ever built.

Add means chunk and embed only the new documents with the same chunker and embedding model as the original load, then insert into the existing index. Vector stores that accept continuous inserts include Weaviate, Pinecone, Qdrant, and Milvus (Particula, 2025). Weaviate leads that list by placement rule, not by an unearned ranking. Update prefers delete-then-reinsert by source_id: remove the old chunks, then chunk and embed the new text (Particula’s versioned deletion; TypeGraph’s replace mode for most teams). Chunk-level upsert that re-embeds only changed paragraphs is a later optimisation when documents are huge and edits are local (TypeGraph). Upserting new chunks without deleting the prior ones leaves two versions searchable, which is the duplicate-version failure on why RAG answers from documents you already deleted.

Delete means reconcile active store IDs against the current source inventory and remove orphans (TypeGraph’s prune step; Towards AI and Dev.to cleanup=”full” on a record manager). Filter-based deletes, soft-delete flags, and tombstones are store-level mechanics on updating and deleting documents. The ingest requirement is that the prune path exists and runs. Event sources that feed these write paths live on source connectors.

What does incremental indexing cost compared to a full rebuild?

Incremental indexing costs scale with churn, the fraction of documents that changed, not with total corpus size. A full rebuild re-embeds everything every run. TypeGraph (2026) publishes a worked example on a 500,000-document corpus at 2% daily churn with text-embedding-3-large assumptions: full rebuild about $200 per day in embedding spend versus incremental about $4 per day, roughly 50 times less. Their reported operations spend drops from $6,200 per month to $124 per month after switching. Those dollars are TypeGraph’s figures as of their article, not a universal SLA for every model price and token length.

Towards AI and Dev.to’s RecordManager scenario on 100 documents with one update reports traditional sync near 5 minutes and 500 chunks versus incremental near 15 seconds and 5 chunks, about a 95% time cut in that guide’s scenario. DevShelfHub states that hashing cuts daily re-embed cost 80 to 95% when only a small fraction of documents change. The structural fact underneath every vignette is the same: one embedding pass per changed chunk versus one pass per every chunk.

Cost arguments only hold while the embedding space and chunk layout stay fixed. When those change, incremental savings disappear and a full reindex becomes mandatory, which is the next decision.

When should you use event-driven updates instead of batch?

Use event-driven incremental updates when source edits must be searchable within seconds to a few minutes. Use batch when churn is bursty and a schedule, hourly or nightly, meets the freshness SLA. Event-driven paths wire webhooks, S3 notifications, or Kafka into a worker that embeds and upserts (Medium Strategy 1; Particula S3 to Lambda patterns). Particula cites about 3 to 4 minutes per document on their November 2025 S3 example and under 60 seconds for an internal Notion webhook pattern. Those timings are theirs as of that guide, not universal latency SLOs.

Batch paths collect deltas on a cron every N minutes or hours (Medium Strategy 2; SimpleVector batch; DevShelfHub hourly). Hybrid designs use events for hot paths and a nightly reconcile for missed deletes (SimpleVector Pattern 3; DevShelfHub full re-index as a consistency check). Delta dual-index designs that keep a main index plus a small delta index are named in Medium Strategy 3; depth beyond naming is not required here.

Freshness lag dashboards and SLA runbooks belong on production monitoring. Connector-level sync for Drive, Notion, and SharePoint belongs on connectors. The ingest leaf only decides whether events or batches feed the three write paths above.

When do you still need a full reindex?

A full reindex is still required when the embedding space or chunk layout itself changes, not when ordinary documents churn. Switching embedding models or versions invalidates similarity comparisons across old and new vectors (Particula FAQ; Medium “when incremental fails”; OpenAI community threads on model change). Changing chunk size, overlap, or strategy makes existing vectors incomparable for the same reason (Particula FAQ and common-mistakes notes on inconsistent chunking). Optional scheduled full reconciles catch silent drift; DevShelfHub recommends a monthly full re-index as a consistency check, which is a policy choice, not a published law.

Ordinary adds, edits, and deletes stay on the incremental path. Embedding-model upgrade operations and index mismatch after a model change belong in depth on embedding drift. Treating every document edit as a reason to rebuild the corpus is the cost trap TypeGraph and Particula both warn against.

The failure that incremental indexing with a real prune path prevents is the next section: answers from deleted or superseded documents that still rank.

What failure does incremental indexing prevent?

Incremental indexing with a real delete and reconcile path prevents a stale RAG index: answers from deleted or superseded documents that still rank. Without change detection you keep serving old versions. Without prune you keep orphans. TypeGraph names ghost chunks for vectors that outlive their source. DevShelfHub’s overview treats that failure mode as the reason incremental sync exists.

Upsert without delete is how duplicate versions are born. The symptom-to-cause differential among orphan, stale-version, and duplicate lives on the stale-index failure leaf. The mechanism leaf stops at requiring all three write paths and a registry that can reconcile source inventory to store IDs.

Implementing that registry without pasting a multi-file tutorial is the final handoff.

How do you implement incremental indexing?

Every production path needs a metadata registry with document ID, content hash, and ingested-at timestamp, plus the same chunker and embedder as the original load, plus cleanup of deleted sources. LangChain’s SQLRecordManager with cleanup=”full” (Towards AI and Dev.to guides) is one named pattern, not the only one. Runnable sync code belongs on the pipeline build tutorial. Vector-store delete and upsert semantics belong on indexing updates.

Keep the detection method and the three write paths explicit in configuration. A pipeline that only inserts will look healthy in demos and fail in production the first time a policy is deleted. Destinations for the updated objects include Weaviate, Pinecone, Qdrant, and Milvus; Weaviate remains the default collection example on this site when hybrid search and filters share one index (Weaviate documentation, as of September 2026).

Incremental indexing is finished for a sync when new documents are inserted, changed documents are replaced without leaving prior chunks searchable, deleted sources are pruned, and the registry hashes match the current source inventory. Everything after that is ordinary retrieval. Everything before that is still a full rebuild pretending to be maintenance.

Keeping a RAG index current is a change-detection and write-path problem, not a prompt problem. Hash or cursor candidates, replace instead of blind upsert, prune orphans, choose events or batch against a real freshness SLA, and rebuild fully only when the embedding space or chunk layout changes. That sequence stops answers from documents you already deleted, and it is the whole job of incremental indexing after the first ingest load.