Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples

This paper proposes a novel method of learning by predicting view assignments\nwith support samples (PAWS). The method trains a model to minimize a\nconsistency loss, which ensures that different views of the same unlabeled\ninstance are assigned similar pseudo-labels. The pseudo-labels are generated\nnon-parametrically, by comparing the representations of the image views to\nthose of a set of randomly sampled labeled images. The distance between the\nview representations and labeled representations is used to provide a weighting\nover class labels, which we interpret as a soft pseudo-label. By\nnon-parametrically incorporating labeled samples in this way, PAWS extends the\ndistance-metric loss used in self-supervised methods such as BYOL and SwAV to\nthe semi-supervised setting. Despite the simplicity of the approach, PAWS\noutperforms other semi-supervised methods across architectures, setting a new\nstate-of-the-art for a ResNet-50 on ImageNet trained with either 10% or 1% of\nthe labels, reaching 75.5% and 66.5% top-1 respectively. PAWS requires 4x to\n12x less training than the previous best methods.\n