Enhancing Nutritional Data Literacy Through an AIDriven Pipeline for Object Detection and Label Interpretation

Abstract

Fundamental limitations in data literacy and manual diet tracking present major barriers to nutritional awareness. While apps like MyFitnessPal enable manual logging and barcode scanning, they fail to recognize unlabeled whole fruits or provide explainable AI interpretations of nutrients-gaps this pipeline addresses through integrated YOLO detection, OCR, and LLM-generated personalized explanations. The system combines computer vision and NLP to extract nutritional information from both packaged labels and whole fruits. OCR captures text from English-language labels with high fidelity, while YOLO-based object detection identifies three fruit types: yellow carabao mango, red fuji apple, and orange Sagada orange. Extracted data is then processed by a Large Language Model that transforms raw numerical values into human-understandable explanations, clarifying nutrient meanings, comparing them with recommended intake thresholds, and offering personalized dietary suggestions. This dual-modality approach allows seamless dietary tracking across packaged foods and unlabeled produce without manual entry. By operationalizing Explainable AI in nutrition, the system reduces cognitive load and supports informed decision-making. The integrated workflow achieved an F1-score of 0.8612 for fruit detection and a Character Error Rate of $\mathbf{2. 1 9 %}$ for OCR accuracy, demonstrating effective technological integration for comprehensive nutritional awareness.