When RAG Returns Another Tenant’s Documents

Cross-tenant RAG leakage is a retrieval that returns chunks whose tenant_id is not the caller’s. The chat reply still looks fluent, because the model paraphrases whatever grounding data arrived, so the failure often surfaces first as a breach report rather than an exception. That is different from a metadata filter that silently excludes the right document inside one tenant, which is why your RAG returns the wrong chunk, and different from within-tenant document ACL depth after tenancy already holds, which is owned by access control. Before any redesign, log the query, the caller’s verified tenant, every retrieved chunk id, each chunk’s tenant_id metadata, and the filters the retriever claims it applied. The failures taxonomy places this leaf among security-shaped retrieval faults sorted by the stage that caused it.

Why does cross-tenant RAG leakage stay silent until someone notices?

Cross-tenant RAG leakage stays silent because retrieval succeeds and generation continues: no ACL error fires, and the model answers confidently from foreign chunks. Shanavas Abdul Samad’s August 2026 production post on physical index isolation states the load-bearing failure mode plainly: one missed tenant filter is not an error, it is a silent success that returns another tenant’s data into an LLM prompt, after which the leaked content is paraphrased into a fluent answer delivered to the wrong tenant’s user. Raza Hussain’s July 2026 Level Up Coding piece frames the same outcome as a breach report rather than a bug ticket when Tenant A asks a question and the system returns Tenant B’s document.

Truto’s 2026 multi-tenant RAG isolation guide rejects filtering restricted data in the LLM system prompt as an architectural anti-pattern, calling it security theater because models are non-deterministic and susceptible to prompt injection. Microsoft Learn’s secure multitenant RAG guide (John Downs and Daniel Scott-Raynsford, architecture center, last updated 21 August 2026) states the positive requirement: only grounding data the user is authorized to access may be incorporated, whether the store is tenant-specific, multitenant, or shared. Burnat’s arXiv paper 2605.19847v2 (31 May 2026), Auditing Privacy in Multi-Tenant RAG under Account Collusion, adds a retrieval-channel fact architecture blogs usually omit: when a service advertises a per-account differential-privacy budget of ε_acc = 1 at δ_acc = 10⁻⁶, a ten-account same-index coalition can leak as if ε were about 3.16, and a fifty-account coalition as if ε were about 7.07, because joint leakage scales as Θ(√k)·ε_acc under Gaussian noise-then-select retrieval. That claim is about the retrieval channel under the paper’s threat model; it is not a measured production cross-tenant leak rate for any named product, and no production leak rate is invented here.

Once the symptom is recognized as foreign tenant_id values in the retrieved set, the next question is which of the five recurring causes produced that set, because each cause has a different fix.

How do you tell which of the five leak causes fired?

Five distinct causes produce the same foreign-tenant symptom, and each has a different fix, so an architecture walk that always starts by provisioning namespaces wastes a sprint when the bug is an unkeyed prompt filter or a stale ACL sync. Differential diagnosis starts from the one symptom and runs the single probe that rules each cause in or out before any storage redesign. Truto, Microsoft Learn, Shanavas, Satadru, and Kevin Patel all teach useful isolation patterns; none of them publishes one table where each row is a discriminating probe that names the cause on a seeded two-tenant corpus.

Read the table as a triage card, not as a ranking of how often each cause appears in your fleet. The probes are boolean gates: you already know which tenant should own every returned chunk. Aggregate security reviews and red-team prompts still matter for fleet health, and Kevin Patel’s LinkedIn write-up on multi-tenant RAG isolation describes unit tests that fail when tenant_id filters are missing, but those checks do not by themselves name which of the five causes fired on one path.

Each row below expands one probe into symptom, cause, and fix. The first cause to check on any shared index that still relies on application filters is a forgotten tenant predicate, because that path fails without an attacker.

What happens when a code path forgets the tenant filter?

A forgotten tenant filter returns another tenant’s documents when isolation lives only in application code, so any path that constructs a vector query without the predicate returns nearest neighbors from the whole shared table. Shanavas (August 2026) lists the concrete load: every new retrieval feature, hybrid path, agent tool, refactor, and debugging script against production must remember the filter for the lifetime of the system, and one miss is silent. Hussain’s Level Up piece states that nearest_neighbors finds the closest vector in the whole table, every tenant’s rows included, unless you scope it yourself, and that application-level scoping catches the happy path while missing the console session, background job, or raw SQL debug query that forgets the where(tenant_id:) clause.

hamurda’s September 2026 Dev.to write-up on scoping a RAG knowledge base per tenant pairs application filtering with Postgres row-level security so the database still refuses foreign rows when application code has a bad day, and makes tenant_id a non-optional parameter on the retrieve function so there is no code path where similarity search runs without it. Kevin Patel’s LinkedIn article on isolating user data in multi-tenant RAG systems describes unit tests that fail if tenant_id filters are missing or misconfigured. Weaviate’s multi-tenancy documentation on weaviate.io (manage-data/multi-tenancy, as of September 2026) states that multi-tenancy provides data isolation, that each tenant is stored on a separate shard, and that data stored in one tenant is not visible to another tenant; multi-tenancy is disabled by default until multiTenancyConfig is enabled on the collection. The same class of storage-scoped isolation appears across Weaviate multi-tenancy, Pinecone namespaces, Qdrant collections, and pgvector with RLS. No product SLAs are invented beyond those published statements. When the filter is present on every path and foreign documents still appear because the caller can rename the tenant, the failure has moved from forgotten filters into untrusted tenant scope.

Why must the tenant ID come from the auth layer and not the request body?

The tenant ID must come from the auth layer because a verified session that says the caller is tenant A is worthless if the retriever trusts a tenant value from the request body, a header, or an LLM-parsed filter expression. Kevin Patel states the rule as injecting tenant context from the authentication or session layer, not from the user, so tampering or impersonation cannot rewrite the filter. Microsoft Learn requires an identity provider to authenticate the requestor, an identity directory that stores authoritative identities, and a mapping from user to tenant so access to that tenant’s data can be granted, with that identity flowing through the request chain to the orchestrator or data store.

hamurda’s chain is concrete: a publishable API key identifies the tenant, the backend issues a JWT containing tenant_id, the database session sets app.tenant_id, and RLS enforces the match, with no point where a user can substitute a different tenant’s ID. Truto’s pool pattern description routes the same way: the API gateway extracts tenant identity from a verified JWT and passes it to the vector database’s fine-grained access control so application bugs cannot append a different tenant_id by hand. When tenant scope is trusted and foreign ids still appear in the candidate list before any post-filter, the architecture is retrieving across tenants first.

What goes wrong when you filter after vector search instead of before?

Filtering after vector search goes wrong when approximate nearest neighbor search runs across a shared index first and the tenant predicate is applied only to the candidate list afterward, so unauthorized vectors have already entered the set that ranking and the model will see. Microsoft Learn states that in multitenant stores, every retrieval request must include a tenant discriminator and any user-level authorization filters, and that those filters should be applied before grounding data is passed to the model. Satadru’s January 2026 Data Engineer Things post on the one-versus-many vector database dilemma describes the required shape as a WHERE clause applied before semantic search: metadata such as domain or tenant is stored at ingest, and similarity runs only inside the filtered subset. Truto insists the deterministic boundary sit at the vector database, populated by an ingestion pipeline that never confuses one customer’s data with another’s, rather than at the LLM layer after candidates already exist.

Satadru also warns that inferring the metadata filter with an LLM before search (dynamic metadata filtering) still requires the filter to be applied as a hard retrieval constraint, not as a suggestion the model may ignore. No timing or existence side-channel leak rates are invented for post-filter designs; the discriminating measurement is the presence of foreign tenant ids before the filter. When the store is correctly scoped and foreign content still reaches the user because the only remaining check is a system-prompt instruction, the failure is prompt-only isolation.

Why is filtering only in the LLM prompt not tenant isolation?

Filtering only in the LLM prompt is not tenant isolation because the model is not a deterministic authorization engine. Truto’s crisis section describes the naive pattern of ingesting every tenant into one index, attaching tenant_id, fetching all relevant documents via vector search, and then instructing the model to use only documents the user may see, and rejects that pattern because models will, with measurable frequency under adversarial prompts, surface chunks they should not have. Truto’s FAQ restates the same rule: never rely on the LLM or system-prompt instructions to filter results; enforce isolation at the vector database query layer. Kevin Patel lists prompt reinforcement (“do not use any content that is not from this client”) as optional and explicitly not a security mechanism.

Prompt text can still name the tenant for clarity once the candidate set is already authorized. It cannot be the only line of defence. When storage scope and trusted tenant identity are both correct and revoked or deleted source documents still retrieve, the failure has moved into permission and delete drift.

What happens when permissions or deletes change but the index does not?

Permissions or deletes that change in the source system while the vector index keeps the old embeddings produce a leak that looks like correct tenancy and still returns content the caller must not see. Truto’s section on deleted records and state drift describes a user deleting a sensitive document in Google Drive while the vector representation remains, so the agent continues to answer from data that no longer exists in the source of truth. The same guide requires mirroring source-system ACLs into chunk metadata at ingest and refreshing them on permission-change webhooks or a permission-only reconciliation job, because ACLs in SaaS systems change constantly and a pipeline that re-syncs ACLs only on document edits will leak for months.

GitHub issue sakibsadmanshajib/hive#1643 (opened 1 September 2026) records a closely related surface mismatch inside one tenant: projects look private because ownership checks refuse another member’s project id, yet public.rag_documents has no uploader column, Repo.SearchChunks is tenant scoped with an optional project filter, and an unscoped search returns the whole tenant corpus including passages from documents attached to another member’s project, with document_name naming the source. The authorization seam correctly refuses a foreign project id and still cannot make the underlying documents unreadable, because they are not owned by anyone below the tenant. That is the “surface claiming a state the system does not have” shape: the UI promise and the RAG store diverge. Running the five probes by hand on every path works for a single incident; packaging them into one suite is what makes isolation checks repeatable across chat, tools, and batch jobs.

How do you prove tenant isolation with one probe suite?

You prove tenant isolation with one probe suite by seeding two tenants with marker documents and running the five discriminating checks against every retrieve entrypoint you ship, then failing the build on any foreign tenant_id. Kevin Patel describes audit logging tagged by tenant id, red-team prompts that ask what another user uploaded, and unit tests that fail when filters are missing. hamurda makes tenant_id non-optional on retrieve so forgetting the parameter is a type or call error rather than a silent default. Those pieces become one function when each block prints which of the five causes fired.

The suite below is a stdlib-only stand-in. Wire retrieve() to your real stack (Weaviate multi-tenancy tenants, Pinecone namespaces, Qdrant collections, or pgvector with RLS) without changing the assertions. The probes are boolean gates on a seeded corpus, not estimates of production leak rate. Isolation bugs do not announce themselves in chat quality metrics; they wait for the worst possible moment.

"""Cross-tenant leakage probes: five causes, one suite (stdlib only).

Wire retrieve() to your real stack before CI.
Each chunk is a dict: {"id": str, "tenant_id": str, "text": str}.
"""

def leak_rate(chunks, caller):
    if not chunks:
        return 0.0
    foreign = sum(1 for c in chunks if c["tenant_id"] != caller)
    return foreign / len(chunks)

def probe_tenant_leakage(*, retrieve, k=5):
    """Print which of the five leakage causes fired.

    retrieve(caller, query, *, tenant_from_body=None, apply_filter=True,
             return_prefilter=False, prompt_only=False) -> list | (pre, post)
    """
    # Cause 1: forgotten filter
    normal = retrieve("acme", "secret roadmap", apply_filter=True)
    open_ = retrieve("acme", "secret roadmap", apply_filter=False)
    if leak_rate(open_, "acme") > 0 and leak_rate(normal, "acme") == 0:
        print("CAUSE 1 forgotten filter -> move tenant scope to storage layer")

    # Cause 2: caller-controlled scope
    spoof = retrieve("acme", "secret roadmap", tenant_from_body="globex")
    if leak_rate(spoof, "acme") > 0:
        print("CAUSE 2 caller-controlled tenant -> derive tenant from verified session")

    # Cause 3: retrieve-then-filter
    pre, post = retrieve("acme", "secret roadmap", return_prefilter=True)
    if leak_rate(pre, "acme") > 0:
        print(f"CAUSE 3 retrieve-then-filter "
              f"(foreign_prefilter_rate={leak_rate(pre, 'acme'):.2f}) "
              f"-> filter-then-retrieve")

    # Cause 4: prompt-only isolation
    promptish = retrieve("acme", "secret roadmap", apply_filter=False, prompt_only=True)
    if leak_rate(promptish, "acme") > 0:
        print("CAUSE 4 prompt-only isolation -> enforce at store or API gatekeeper")

    # Cause 5: permission / delete drift
    revoked = retrieve("revoked-user", "board minutes", apply_filter=True)
    if any(c.get("acl_allows_revoked") or c.get("source_deleted") for c in revoked):
        print("CAUSE 5 permission or delete drift -> fail-closed sync + hard deletes")

    if leak_rate(normal, "acme") == 0 and leak_rate(spoof, "acme") == 0:
        print("baseline retrieve as acme: PASS (no foreign tenant_id)")

The suite is deliberately transparent: each block is one row of the detection table, so you can delete causes you have already ruled out. Keep failing queries and marker corpora in a a golden test set so the next deploy cannot resurrect a path that once leaked. Cross-tenant leakage is not a mysterious model failure. It is one of five retrieval-authorization faults: a forgotten filter, an untrusted tenant scope, a post-filter shared ANN pass, a prompt pretending to be a boundary, or a stale permission and delete sync, each isolated by one probe and fixed by one matching change at the store, the auth layer, or the sync pipeline.