Large language models have improved conversational AI by producing fluent, context-aware replies. Still, their short context windows and lack of persistent memory can break conversations: users often must repeat important details. This gap harms personalization and makes it hard for systems to keep a consistent, long-term dialogue. This paper tackles that problem by developing memory extraction and retrieval methods to capture and preserve important user information from conversations. The proposed system extracts compact facts—user preferences, notable events, and situational details—and stores them in a structured repository. Those facts are converted into semantic vectors through modern vectorization (embedding) techniques, which enable fast, similarity-based lookup. During a conversation, the system retrieves relevant memory vectors and injects the matching information into the model’s context. That dynamic retrieval enriches the model’s input with past, relevant details, improving response coherence and personalization without changing the language model itself. Evaluation shows that this memory-augmented pipeline effectively adds high -similarity and usable context to the models.
Paper
Full text
Expanding Conversational AI Beyond Context Windows: A Memory-Augmented Approach
Semantic Scholar · 2025
Abstract
Large language models have improved conversational AI by producing fluent, context-aware replies. Still, their short context windows and lack of persistent memory can break conversations: users often must repeat important details. This gap harms personalization and makes it hard for systems to keep a consistent, long-term dialogue. This paper tackles that problem by developing memory extraction and retrieval methods to capture and preserve important user information from conversations. The proposed system extracts compact facts—user preferences, notable events, and situational details—and stores them in a structured repository. Those facts are converted into semantic vectors through modern vectorization (embedding) techniques, which enable fast, similarity-based lookup. During a conversation, the system retrieves relevant memory vectors and injects the matching information into the model’s context. That dynamic retrieval enriches the model’s input with past, relevant details, improving response coherence and personalization without changing the language model itself. Evaluation shows that this memory-augmented pipeline effectively adds high -similarity and usable context to the models.