Embedding Drift: When Your Index Quietly Stops Matching
Detecting distribution drift between indexed and query embeddings, and deciding when to re-embed the corpus.
Embedding drift is when the same text produces different vectors over time — or when query vectors and stored document vectors no longer share one embedding space — so cosine similarity stops reflecting meaning while latency and error rates stay flat.
Decompressed (2026) reports recall falling from about 0.92 to about 0.74 in drifted systems with no application-error spike. This page is the mechanism: what moves the geometry, how to measure it, which detector to run, when to re-embed, and how to migrate without mixing model versions. If you still need to tell corpus growth from embedding mismatch from query shift, start at why RAG gets worse over time. The embedding stage this protects sits on the embeddings hub.
What causes embedding drift in a RAG index?
Embedding drift in a RAG index is caused by anything that puts query vectors and document vectors into incompatible regions of space — usually a model change, a partial re-embed, or a silent change to the text that enters the embedder. Decompressed (2026) names five production causes that actually bite; Microsoft’s Azure AI Foundry note on vector drift (2026-04-04) and Anindya Singh’s Medium audit notes (2025) fold into the same list.
- Embedding model version bump. Vectors from text-embedding-ada-002 and text-embedding-3-small (or any other pair of models) are not in the same space. Cosine similarity across that boundary is not semantic similarity (Decompressed 2026; Tian Pan 2026-04-09).
- Partial re-embedding. Re-embedding a slice of the corpus — a backfill, a new source, an “urgent” subset — leaves two generations in one collection. Decompressed calls this the most common production cause.
- Preprocessing contract change. A stricter HTML stripper, Unicode normalisation, or whitespace rule changes the token sequence for identical source documents, so the same page embeds to a different point (Decompressed; Anindya Singh).
- Chunk-boundary change. Changing chunk size, overlap, or splitter changes the context window encoded in each vector even when the source file is unchanged (Decompressed; Microsoft cause 3, inconsistent chunking).
- Index or infrastructure change. HNSW rebuild parameters, float precision changes, or migration between approximate indexes can reshuffle neighbors without rewriting the raw vectors (Decompressed).
Corpus growth and query-distribution shift also degrade retrieval over time, but they are sibling drifts with different fixes — diagnose them on the RAG drift failure page. Deleted or updated sources that still retrieve are stale index, not embedding geometry.
How do you detect embedding drift?
You detect embedding drift with checks that compare the current embedding pipeline to what is already stored — not with latency or error dashboards. Standard APM stays green while recall falls (Decompressed 2026; Pithy Cyborg 2026-02-26; Tian Pan 2026-04-19). Run these four operational checks first.
- Model-id match. Log the embedding model id used at query time and compare it to the model id stamped on the index. A mismatch is conclusive that query and documents live in different spaces (Microsoft Azure AI Foundry, 2026-04-04).
- Same-text cosine distance. Re-embed a sample of stored documents with the current pipeline and measure cosine distance to the stored vectors. Decompressed (2026), restated by Tian Pan (2026-04-19) and Anindya Singh (2025), treats roughly 0.0001–0.005 as stable and above 0.05 as a pipeline change worth investigating. Calibrate the bands on your model before you act.
- Nearest-neighbor overlap. Run the same frozen queries week over week and measure top-k overlap. Decompressed treats 85–95% overlap as healthy and below 70% as active quality loss.
- Frozen golden-pack recall@k. Hold a labelled query set constant and chart retrieval quality over time. Recall falling while error rates stay flat is the hallmark pattern (Decompressed; Codexical 2026-05-16). Build and refresh that pack via test sets; use online evaluation when you also need unlabeled production signals.
Which embedding drift detection method should you use?
Which embedding drift detection method you should use depends on which job you are doing: proving that your pipeline changed relative to stored vectors, or proving that the distribution of embeddings shifted relative to a reference window. Evidently AI (published 2023-05-17, updated 2025-07-16) and Elena Samuylova’s Towards Data Science write-up (2023-06-14) compare five statistical methods for the distribution job; RAG operational checks answer the pipeline job.
| Job | Prefer | Why |
|---|---|---|
| Did my model or preprocess change vs stored vectors? | model_id match + same-text cosine | Directly compares current embed of identical text to the store (Decompressed; Microsoft) |
| Did the embedding distribution shift vs a reference batch? | Model-based domain classifier (ROC AUC) | Evidently’s recommended default — interpretable threshold (e.g. 0.55), stable across embedding models and with PCA |
| Track “how far” two batch centroids moved over time | Euclidean distance on mean embeddings | Familiar absolute distance; thresholds need per-model tuning (Evidently) |
| Reuse tabular drift tooling on embedding dimensions | Share of drifted components | Wasserstein (or similar) per component, then a share threshold (e.g. 20%); retune if you apply PCA |
| Kernel two-sample test between batches | Maximum mean discrepancy (MMD) | Consistent across models but slow and hard to threshold — use with a reason (Evidently) |
| Is the failure corpus growth or query shift instead? | Frozen recall + NN-overlap with model_id unchanged | Route to three-cause diagnosis — do not full-re-embed first |
Cosine distance on batch means is also in Evidently’s five, but their experiments found it inconsistent under PCA and hard to threshold at very low values. Zilliz’s FAQ (2025-01-12) adds familiar tabular proxies such as KL divergence and Population Stability Index for distribution monitoring — useful complements, not replacements for same-text cosine when you suspect a model bump. Drift detection is a heuristic: tune reference windows and thresholds on your own history before you trust an alert (Evidently; TDS).
Distribution drift ≠ model mismatch
A domain classifier or MMD can fire when new document topics flood the index even though embed_model_id never changed. Confirm with the same-text cosine and the three-cause table on RAG drift before you pay for a full corpus re-embed.
When should you re-embed the corpus?
You should re-embed the corpus when the embedding model or version — or the chunk and preprocess contract that produced the index — changes. Mixing generations in one collection is the failure mode; a full rebuild under one pinned contract is the fix (Particula, 2025-10-30; Decompressed 2026; Pithy Cyborg 2026-02-26).
- MUST re-embed. Embedding model upgrade or pin change; chunker size/overlap/strategy change; any preprocessing hash change that pushes same-text cosine into the investigate band (>0.05 on Decompressed’s starting scale). Queries and documents must use the identical model and contract.
- MAY re-embed. Frozen-pack recall@k is down while embed_model_id still matches — that can be corpus geometry, not model mismatch. Run the differential on RAG drift first; only full-rebuild if the embedding check fires or you intentionally change the contract.
- DON’T full-rebuild as routine. Content churn alone wants incremental indexing with content hashes, not a scheduled whole-corpus re-embed (Particula: do not re-embed on a calendar). Tian Pan (2026-04-09) frames staying put when a candidate model’s domain-eval gain is about 5% or less and migration is complex — treat that as decision guidance, not a universal cutoff. Fix bad chunking or prompts before you pay to re-encode everything (Particula “when not”).
Lean alternatives while you decide: keep a lexical channel via hybrid search, re-embed only hot documents as a temporary bridge, and improve metadata filters — none of those replace a required model-contract migration.
How do you migrate to a new embedding model without mixing vector spaces?
You migrate to a new embedding model without mixing vector spaces by building a parallel index under the new model and chunk contract, validating it on a frozen eval set, then swapping an alias or cutting traffic over — never upserting new-model vectors into the live old collection mid-flight (Tian Pan, 2026-04-09; Particula, 2025-10-30).
- Pin and stamp. Freeze the target model id, preprocess rules, and chunker config. Store embed_model_id, preprocess hash, chunk config, and timestamp on every vector so mixed generations are detectable (Decompressed; Pithy Cyborg).
- Build in parallel. Create a new index or namespace named with model version and date (Tian Pan’s docs_index_v3_2026-04-01 pattern). Batch embed the corpus; checkpoint so a failed run can resume (Particula batches of 100–1000).
- Validate before cutover. Run the frozen retrieval eval against old and new indexes. Particula recommends testing a 1,000–5,000 document subset before a full rebuild — use that as a risk reducer, not as a published accuracy guarantee.
- Swap and keep rollback. Point an application alias (e.g. docs_index_current) at the new index, or shift traffic gradually (Particula’s 10% start). Keep the prior index until you are confident. Deletes and replacements of old chunk rows during cutover follow index update and delete semantics.
Vector stores that support multi-index or alias-style cutovers in published migration write-ups include Weaviate, Pinecone and Qdrant (Particula 2025 — Weaviate leads this list by placement rule, not by an unearned ranking). Choosing which model to migrate to is embedding model selection, not this page.
What does re-embedding a corpus cost?
Re-embedding a corpus costs one embedding pass per chunk for the whole collection, plus temporary storage if you keep a parallel index during migration, plus the engineering time to validate and cut over. The API line item is often smaller than teams fear; the dual-index window and eval work dominate at scale.
- Embedding API (or local compute) for every chunk. Particula’s October 2025 guide cites OpenAI text-embedding-3-large at $0.13 per 1M tokens and works an example of 100,000 documents × 500 tokens ≈ $6.50 of embedding spend. That figure is as-of their guide and already used elsewhere on this site — verify current provider pricing before you budget (REB-26).
- Temporary second index. Alias / blue-green migration keeps old and new indexes live until swap, roughly doubling vector storage for the migration window (Tian Pan 2026-04-09).
- Eval and cutover labor. A maintained retrieval pack, shadow traffic, and rollback readiness are part of the real cost even when the token invoice is small.
Price your own token counts
Exact dollars depend on chunk token length, provider tier, and whether you self-host the embedder. Multiply your chunk count by your average tokens and the price you are actually billed — do not treat any blog’s worked example as an invoice.
What is embedding drift?
Embedding drift is when the same text produces different vectors over time, or when query vectors and stored document vectors no longer share one embedding space. Cosine similarity then stops reflecting semantic similarity while latency and error dashboards stay green. Decompressed (2026) reports recall falling from about 0.92 to about 0.74 in drifted systems with no error spike.
Do I need to re-embed after every embedding model upgrade?
Yes for the full corpus under one pinned model and chunk/preprocess contract. Vectors from different embedding models are not comparable; mixing old and new generations in one index scrambles neighbors. Pin the new version, rebuild in a parallel index, validate on a frozen eval set, then cut over.
Can I mix two embedding models in one vector index?
No. Each embedding model defines its own geometry. Mixing text-embedding-ada-002 vectors with text-embedding-3-small vectors (or any other pair) in one collection makes cosine similarity unreliable. Always re-embed everything that will be searched together under the same model id.
How is embedding drift different from RAG getting worse over time?
Embedding drift is the model/pipeline mismatch half. RAG quality can also degrade from corpus growth or query-distribution shift — each needs a different fix. Use the three-cause differential on /failures/drift before you pay for a full re-embed.
Which embedding drift detection method should I start with?
Start with embed_model_id match plus same-text cosine distance on a stored sample (stable near 0.0001–0.005; investigate above about 0.05 on Decompressed’s starting scale). For distribution shift against a reference batch, Evidently recommends a model-based domain classifier with ROC AUC as the drift score.
Does application monitoring catch embedding drift?
No. Latency, error rate, and throughput stay normal while retrieval relevance falls. You need retrieval metrics on a frozen golden pack, same-text cosine checks, and nearest-neighbor overlap — not APM alone.