Agentic Retrieval for Conversational Memory in LLM-Based Chatbots

Large language model (LLM)–based conversational systems are constrained by fixed context windows, limiting their ability to retain and retrieve information from extended dialogue histories. While context window expansion and memory injection approaches have been proposed, they incur significant computational costs and lack mechanisms for selective, intent-aware recall. Existing retrieval-augmented generation (RAG) systems are primarily designed for document retrieval and fail to address the structural and temporal characteristics of conversational memory. This paper presents an agentic RAG system that overcomes these limitations through dual retrieval pathways tailored to different query intents. A query routing agent directs exploratory queries to topic-based cluster navigation and routes targeted queries to hybrid semantic-lexical retrieval over exchange-based conversational units with LLM-powered context enrichment. Topic discovery is performed automatically using UMAP dimensionality reduction and HDBSCAN clustering without predefined categories. Evaluation on real-world conversational data demonstrates strong performance, including high query classification accuracy and perfect hit rates with high recall for targeted queries. The system integrates with an open chatbot interface (Open WebUI), supports multi-user deployment with access control, and operates without model fine-tuning.