Employing Knowledge Distillation to Create Light-Weight Lane Detection Machine Learning Models for Low-Cost Computing Environments

Abstract

Deep learning (DL) is a prominent and growing area of machine learning that is rapidly changing how complex data in the world is modeled, interpreted, and utilized. To model these complexities, building accurate and robust DL models can often require heavy computation, memory, and energy costs. Because of this, researchers have been exploring methods to construct deep neural networks with small model sizes, light computation costs, and high segmentation accuracy. Knowledge distillation, a process in which a larger, cumbersome model is "distilled" into a smaller, student model, is a model compression method growing in popularity. In this research, we design efficient and compact deep networks using knowledge distillation to be applied to lane detection in autonomous vehicles with a low-cost computing environment in a classroom, such as Raspberry Pi. We address this by developing a student Convolutional Neural Network that reduces model size and minimizes accuracy loss as measured by the differences in MSE and R2 between the cumbersome model and the student model. Our research suggests that knowledge distillation can be applied to deep learning models trained for lane detection with improved model compactness and moderate accuracy preservation.