Resolution-Based Distillation for Efficient Histology Image Classification

Developing deep learning models to analyze histology images has been\ncomputationally challenging, as the massive size of the images causes excessive\nstrain on all parts of the computing pipeline. This paper proposes a novel deep\nlearning-based methodology for improving the computational efficiency of\nhistology image classification. The proposed approach is robust when used with\nimages that have reduced input resolution and can be trained effectively with\nlimited labeled data. Pre-trained on the original high-resolution (HR) images,\nour method uses knowledge distillation (KD) to transfer learned knowledge from\na teacher model to a student model trained on the same images at a much lower\nresolution. To address the lack of large-scale labeled histology image\ndatasets, we perform KD in a self-supervised manner. We evaluate our approach\non two histology image datasets associated with celiac disease (CD) and lung\nadenocarcinoma (LUAD). Our results show that a combination of KD and\nself-supervision allows the student model to approach, and in some cases,\nsurpass the classification accuracy of the teacher, while being much more\nefficient. Additionally, we observe an increase in student classification\nperformance as the size of the unlabeled dataset increases, indicating that\nthere is potential to scale further. For the CD data, our model outperforms the\nHR teacher model, while needing 4 times fewer computations. For the LUAD data,\nour student model results at 1.25x magnification are within 3% of the teacher\nmodel at 10x magnification, with a 64 times computational cost reduction.\nMoreover, our CD outcomes benefit from performance scaling with the use of more\nunlabeled data. For 0.625x magnification, using unlabeled data improves\naccuracy by 4% over the baseline. Thus, our method can improve the feasibility\nof deep learning solutions for digital pathology with standard computational\nhardware.\n