Revolutionizing PDF Q&A with Local LLMs and Privacy-Enhanced Retrieval-Augmented Generation

With the rising prominence of Large Language Models (LLMs) in natural language processing, our research explores a novel approach to PDF-based question-answering (QA) using Retrieval-Augmented Generation (RAG). Traditional keyword-based QA methods, while effective at basic information retrieval, struggle with contextual understanding and synonym recognition. Subsequently, deep learning models improved these aspects but faced challenges in providing structured answers. The introduction of LLMs addressed many of these limitations; however, they introduced issues like hallucination and high costs for API-based implementations. Our study proposes a cost-effective, privacy-respecting solution that leverages Ollama to run LLMs locally, eliminating the need for cloud-based API calls and mitigating data privacy concerns. Additionally, extensive testing on a structured dataset demonstrates improved accuracy and relevance in answers, making our solution well-suited for secure environments where data sensitivity and cost-efficiency are paramount. This paper contributes a scalable, secure architecture for document-based QA, providing a comprehensive alternative to traditional and cloud-dependent methods.