Visual Content Detection in Educational Videos with Transfer Learning and Dataset Enrichment

Lecture video is transforming education by supplementing and replacing classroom teaching. Recent research has focused on enhancing information retrieval for video lectures with advanced navigation, searchability, summarization, and QA chatbots. Visual elements like tables, charts, and illustrations are central to comprehension, retention, and data presentation. These lecture video elements are i) typically artificially created without a standardized structure, ii) lack clear boundaries, and iii) may be composed of connected text and visual components. Because of the unique nature and scarcity of annotated datasets, current deep learning-based object detection models do not yield satisfactory performance on lecture video frames. This paper reports on a transfer learning approach to this problem. A suite of state-of-the-art object detection models were evaluated for their performance on lecture video datasets. YOLO emerged as the most promising model for this task. Subsequently YOLO was optimized for lecture video object detection with training on multiple benchmark datasets and deploying a semi-supervised auto-labeling strategy. Results evaluate the success of this approach, also in developing a general solution to the problem of object detection in lecture videos. Paper contributions include a publicly released benchmark of annotated lecture video frames, along with the source code to facilitate future research.11https://github.com/dipayan1109033/edu-video-visual-detection22https://github.com/dipayan1109033/LVVO_dataset