Ragas: Automated Evaluation of Retrieval Augmented Generation

Project Description This archive contains a research artifact for the design, implementation, and evaluation of a Retrieval-Augmented Generation (RAG) system in the domain of application security. The artifact provides a complete, self-contained snapshot of the system architecture, curated evaluation data, and automated benchmarking pipelines. RAG System Architecture The archive includes a fully implemented RAG pipeline designed to ground large language model outputs in authoritative security documentation. The system follows a modular architecture with clearly separated components for document ingestion, semantic retrieval, and answer generation. The knowledge base is derived from OWASP Cheat Sheet Series and related security guidance, enabling security-focused question answering with reduced risk of hallucinated responses. Ground Truth Dataset A ground truth dataset is included to support systematic evaluation. The dataset consists of security-relevant questions and corresponding reference information derived from OWASP materials. It is intended exclusively for offline evaluation and benchmarking of RAG behavior and is not meant to serve as training data.