RAG vs AI Memory: Document Retrieval or Per-User State?

Shared corpus retrieval is RAG; per-user evolving state is memory; production agents usually need both. Retrieval-augmented generation fetches document chunks that are similar to a query from an index the organisation controls. AI memory stores what a specific user or agent has said, preferred, and decided across turns, then scores that state by more than similarity alone. Mem0’s RAG versus AI memory article (captured 18 September 2026) states the split as relevance-of-content versus relevance-of-user, and Memori Labs (captured 18 September 2026) frames memory as a persistent context store the agent can read, write, and update. Among the options for choosing between the options, this comparison draws the retrieval-side boundary so a document index is not mistaken for user state.

How does document retrieval differ from per-user memory?

Document retrieval answers “what passage is most similar to this query,” while per-user memory answers “what should this agent remember about this user or session.” Mem0 describes RAG as semantic search with extra steps: fetch external chunks, inject them into the context window, generate. Supermemory’s memory versus RAG docs (captured 18 September 2026) warn that treating memory as another retrieval problem fails because memory is about relationships, time, and importance, not only finding similar text.

Memori Labs calls RAG a retrieval layer that injects external knowledge dynamically at query time, solving knowledge cut-off and access to private corpora. The same article defines memory as persistent state across interactions so an agent can say what happened last week without the user re-feeding the conversation. Similarity search over chat logs is still RAG-shaped storage; it becomes memory only when write, update, conflict resolution, and decay are first-class.

Oracle’s from-RAG-to-memory architecture notes (captured 18 September 2026) sit in the same family: moving from stateless retrieve-then-generate toward stateful agents that keep working context. AWS Builder’s concepts piece in the same harvest treats that shift as an architecture change, not a rename of the vector index. The boundary rule for builders is ownership: shared handbooks and wikis belong in RAG; preferences, allergies, prior decisions, and session goals belong in memory.

When the product is a Q&A bot over a company corpus with no personalisation, memory is optional. When the product is an agent that talks to the same person repeatedly, memory is the layer that stops the system from recommending products the user already refused, which is the failure Mem0 recounts when RAG scores looked high while user relevance was wrong.

What does RAG store compared with what memory stores?

RAG stores documents, chunks, embeddings, and metadata for a shared or permissioned corpus. Memory stores user-scoped facts, episodes, preferences, and summaries that change as conversations continue. Supermemory distinguishes documents as raw static knowledge that does not change based on who accesses them from memory entries that are about the agent-user relationship over time.

Mem0’s technical contrast is blunt: RAG ranking is primarily similarity-based, optionally with a reranker still asking what content is closest to the query. Memory scoring combines recency, importance, and similarity so that “allergic to peanuts” can outrank “likes jazz” when the jazz memory is merely closer in embedding space. That scoring function is why dumping chat transcripts into Weaviate, Pinecone, or Qdrant without importance and conflict rules is still a RAG pattern wearing a memory label.

Write paths diverge too. RAG ingestion is an organisational pipeline: chunk, embed, index, re-index when the handbook changes. Memory ingestion is continuous and personal: each turn may add, update, or delete a fact after conflict resolution. Mem0 describes an LLM-based resolver that decides whether a new statement should create, update, delete, or leave alone an existing memory when semantic similarity detects a clash.

Permission models differ in practice even when both sit in the same database product. A shared policy corpus is usually role-scoped across many users. A memory store is user-scoped by default and must not leak one tenant’s preferences into another’s retrieval. Hybrid stacks keep those collections separate even when both are vectors.

When do agents need memory on top of RAG?

Agents need memory on top of RAG when continuity across turns is part of the product, not when the only gap is missing documents. Memori Labs states that RAG alone is not enough because retrieval does not evolve: an agent cannot say “last week you told me” unless that conversation was manually re-fed. Personalisation by user ID filters on a document index is not the same as learning from behaviour.

Mem0’s when-to-use-memory guidance clusters around multi-turn assistants, support agents that must respect prior constraints, and any flow where forgetting a preference creates harm or trust loss. RAG still owns the handbook, the SKU catalogue, and the policy PDF. Memory owns the customer’s stated constraints and the agent’s prior commitments.

Leonie Monigatti’s from-RAG-to-agent-memory notes (captured 18 September 2026) and Vectorize Hindsight’s developer contrast sit in the same harvest as practitioner framing: agents that plan and act over time outgrow stateless retrieval. The architecture depth for when retrieval itself becomes a decision among tools lives under when retrieval becomes a decision; this comparison only decides whether user state is missing.

Skip memory when every session is anonymous, when answers must come only from a governed corpus with no personalisation, or when you are still failing basic retrieval quality. Adding a memory product will not fix wrong chunks. Fix the index first, then add state for the users who return.

What types of memory show up beside RAG?

Short-term, long-term, and episodic memory labels show up beside RAG as ways to group what gets written and how long it lives. Memori Labs lists multiple implementation styles for memory, from conversation buffers to longer-lived stores, without treating those labels as substitutes for a document index. Short-term memory usually means the working context of the current session; long-term memory means durable facts that survive sessions; episodic memory means event-like records of what happened.

Those taxonomy words are lean here because the decision does not turn on the label. It turns on whether the record is shared corpus evidence or user-scoped state. A “long-term memory” collection that only embeds PDFs is still RAG. A “short-term” buffer that updates a user’s constraints after each turn is memory even if it lives in the same vector product.

Oracle and AWS Builder materials in this harvest emphasise stateful architecture over naming contests. Prefer operational questions: what is written, who owns it, how conflicts resolve, and when entries decay. Mem0’s forgetting discussion exists because unbounded memory stores become noisy; decay and conflict resolution are part of the memory job, not optional polish.

When taxonomy debates start to dominate design meetings, return to the ownership table. Shared corpus to RAG. Per-user evolving state to memory. Hybrid when both are real. The labels then become filing cabinets inside that rule rather than competing architectures.

How do RAG and memory overlap in hybrid agent stacks?

Hybrid stacks retrieve shared knowledge with RAG and load user state with memory in the same turn, then generate once. Mem0’s production hybrid pattern initialises a memory layer for user-specific state alongside a document store for corpus chunks, which is the shape Memori Labs calls the RAG plus memory hybrid. Neither side replaces the other: memory without RAG lacks external factual grounding, and RAG without memory forgets the user.

Memori Labs states that memory alone is not enough because experience stores may lack external factual grounding. A remembered preference does not invent the current refund policy. The policy still comes from the index. The preference comes from memory. Production agents that skip either half produce fluent answers that are either impersonal or ungrounded.

Ordering in the prompt usually puts durable user constraints near the instruction and retrieved passages near the evidence block, but the exact packing is an implementation detail. What matters for the decision is separate write paths and separate evaluation: score retrieval on recall and precision over the corpus; score memory on whether the right user facts survive and update without leaking across tenants.

Vendor APIs for memory products are intentionally lean mentions here. The retrieval-side rule does not require adopting any one memory vendor, and it does not redefine RAG as obsolete. Agents that retrieve then act still need document retrieval when the question is about the corpus; they need memory when the question is about the person.

Naresh Kancharla’s Medium note in the same harvest (captured 18 September 2026) is a short restatement of the same confusion: developers reach for RAG when the symptom is forgetting a user, then wonder why similarity search keeps surfacing the wrong personal fact. The fix is not a larger top-k. The fix is a memory write path with importance and conflict rules beside the shared corpus index.

Evaluation should stay split even in hybrid demos. Corpus questions score against labelled passages. Memory questions score against whether the agent retained, updated, or correctly forgot a user fact. Mixing those labels into one “answer quality” score hides which store failed. Mem0’s peanut-allergy example is memorable because similarity looked fine while user relevance was wrong; your metrics must be able to catch that class without blaming the embedding model alone.

RAG and memory solve different ownership problems. Put shared, permissioned documents in a retrieval index; put per-user evolving state in a memory store with write, conflict, and decay rules; combine them when agents must be both grounded and continuous. Mem0, Memori Labs, Supermemory, Oracle, and AWS Builder converge on that split even when their product pitches differ, and the decision stays on the retrieval side of the line rather than on renaming the index as memory.