Co-Training with Active Contrastive Learning and Meta-Pseudo-Labeling on 2D Projections for Deep Semi-Supervised Learning

A major challenge that prevents the training of deep learning models is the limited availability of large quantities of accurately labeled data. This shortcoming is particularly acute in areas such as medical and biological sciences where data annotation is an expert-demanding, time-consuming, and error-prone undertaking. In this regard, semi-supervised learning tackles this challenge by capitalizing on scarce labeled and abundant unlabeled data; however, state-of-the-art methods typically depend on pre-trained features and large validation sets to learn effective representations for classification tasks. In addition, the reduced set of labeled data is often randomly sampled, neglecting the selection of more informative samples. Here, we present active Deep Feature Annotation (active-DeepFA), a method that effectively combines contrastive learning, teacher-student-based meta-pseudo-labeling and active learning to train non-pre-trained CNN architectures for image classification in scenarios of scarcity of labeled and abundance of unlabeled data. It integrates deep feature annotation (DeepFA) into a co-training setup that implements two cooperative networks to mitigate confirmation bias arising from pseudo-labels. The method starts with a reduced set of labeled samples by warming up the networks with supervised contrastive learning. Afterward and at regular epoch intervals, label propagation is performed on the 2D projections of the networks' deep features. Next, the most reliable pseudo-labels are exchanged between networks in a cross-training fashion, while the most meaningful samples are annotated and added to the labeled set. The networks independently minimize an objective loss function comprising supervised contrastive, supervised, and semi-supervised loss components, enhancing the representations for image classification. Our approach is evaluated on seven challenging image datasets across three distinct domains with only 5% of labeled samples, surpassing baselines and outperforming seven established benchmarks. In addition, it reduces annotation effort by achieving comparable results to those of its counterparts with only 3% of labeled data.