A Frugal Hybrid Architecture for Local AI: Marrying Tiny Recursive Models and External Memory

Preprint (not peer reviewed). This record provides a citable version and establishes precedence via Zenodo DOI: 10.5281/zenodo.18055127. No experimental results are provided; the manuscript proposes an architecture and reproducible implementation/evaluation protocols. Abstract State-of-the-art Large Language Models (LLMs) require substantial computational resources. Nevertheless, a compact LLM can run on a standard consumer hardware. With a more constrained parameter budget (on the order of about 7B), such a model can still leverage very large corpora that encode a substantial fraction of explicit textual knowledge, at the cost of reduced processing capacity and a higher risk of hallucinations. Recent work has shown that another class of neural networks, Tiny Recursive Models (TRMs), can achieve, on certain abstract reasoning benchmarks, reasoning performance close to that of very large LLMs while using only a few million parameters. However, these models remain insufficient to cover very broad natural language domains (for example, open-domain text interactions) and are instead better suited to bounded domains (structured tasks, puzzle solving, etc.). We describe an architecture in which an enlarged TRM (≈ 10^8–10^9 parameters) is coupled with a compact LLM (about 7B) acting as a high-capacity semantic memory. Our goal is to explore whether, on common textual queries, such a TRM–LLM architecture can approximate some of the performance of larger LLMs while keeping cost and latency compatible with consumer hardware. This paper presents a typology of external memories (RAG and/or frozen LLM) and their preparation, the TRM’s decision policy for accessing them, the formulation of queries, and the retrieval/reranking of passages. We also outline several more exploratory directions, such as moving from token-level exchange to embedding-level exchange, and using offline computational “sleep” phases to optimize the exchange hyperparameters between the TRM and its external memory as a function of context, together with selective consolidation using adapters or LoRA, of verifiable TRM acquisitions into a frozen LLM treated as long-term high-capacity semantic memory (frozen LLM only). An appendix describes a plausible implementation (settings and safeguards) intended to foster reproducibility, without in itself claiming to constitute an experimental evaluation. The main contributions are: (i) the specification of a frugal local TRM+LLM/RAG architecture targeting PCs or embedded robots, (ii) a formulation/retrieval protocol with traceability and operational safeguards, and (iii) an exploratory framework for embedding-level exchanges and computational “sleep” phases (offline self-optimization of exchange hyperparameters and selective consolidation into a local LLM). We emphasize that no experimental results are provided here; the paper instead aims to provide a reproducible template open to discussion. Author background: former Chief Scientist at Fondation Internet Nouvelle Génération (FING). Keywords: Tiny Recursive Models (TRM); Large Language Models (LLM); Retrieval-Augmented Generation (RAG); external memory; hybrid architecture; local AI; edge AI; frugal AI; gating policy; reranking.