Improved Semantic Segmentation With Large-Scale Attention-Based Self-Supervised Few-Shot Learning

Abstract

Training deep learning models requires large and diverse datasets. However, some fields, such as medical imaging and defect detection, encounter data collection and labeling issues, privacy concerns, and high annotation costs. This paper presents an unsupervised learning method combined with the attention mechanism to optimize the classification model for domain adaptation. Based on an autoencoder architecture, the attention mechanism ensures that teacher and student models focus on the same features within the same image during simultaneous training, thus improving classification accuracy. The proposed approach reduces overfitting by learning homogeneity and heterogeneity among categories, which enhances the model’s generalization on limited labeled data through few-shot learning. Experimental results show that our method improves segmentation accuracy by 20% mIoU over the baseline method on benchmark datasets, demonstrating its effectiveness in few-shot segmentation tasks.