LLMs with Memory: A Two-Pillar Architecture — Store/Index Separation and Sleep-Time Consolidation

Current large language models forget once a conversation closes: the next session does not know its interlocutor. This paper addresses a single question—how to give an LLM memory—and answers it with one structure rather than a new model. The central claim is that memory divides into two pillars, each struck by its own mechanism. The base-LLM pillar (general knowledge, folded into weights at pretraining) is struck by the prompt; the RAG-store pillar (the individual's full-text conversation history) is struck by a semantic index. The index is not a layer but a striking rod, and the two pillars stand side by side, merging at inference. From this follows the paper's core design decision: the store and the index are separated, and compression is applied only to the index. The RAG store holds the full text and is append-only—never summarized, never deleted—so nothing essential (conditions, exceptions, prohibitions) is lost. The index is built with UDSC (Universal Dense Semantic Compression), whose lossy, proposition-only representation—ordinarily a weakness for body text—becomes optimal for an index. This separation reconciles two requirements usually in tension: "never forget" (the store deletes nothing) and "forgetting is necessary" (the index backgrounds rarely used memories by recall-weighting, without deletion). It also explains why the barb of exact wording need not persist in the foreground: the manner of speaking drops from the index while remaining recoverable from the store. The paper further frames memory movement across a hierarchy (ultra-short-term / short-term / mid-term / long-term / immovable store) organized by the phase in which rewriting is possible, and locates organizing and consolidation in a "sleep" (non-response) phase. Crucially, sleep is argued to be not an essential constraint but an implementation choice under resource limits: given sufficient resources, or once the index is specialized (a CPU-to-ASIC move that also applies to RFAIF's internalization), organizing and consolidation can run in real time, rendering sleep unnecessary. UDSC (DOI: 10.5281/zenodo.21372390) supplies the index language; RFAIF (DOI: 10.5281/zenodo.21094989) supplies the consolidation method that internalizes recurrent, stabilized behavior into weights at zero inference-time token cost. The paper is a design proposal; a working prototype is not yet complete, though the UDSC index can already be operated in substitute form by a general-purpose LLM. Persona construction from retained memory, and the external "spontaneity" program whose injected openings become origin-indistinguishable from user input, are explicitly deferred to future work.