TAFSSL: Task-Adaptive Feature Sub-Space Learning for few-shot classification

The field of Few-Shot Learning (FSL), or learning from very few (typically\n$1$ or $5$) examples per novel class (unseen during training), has received a\nlot of attention and significant performance advances in the recent literature.\nWhile number of techniques have been proposed for FSL, several factors have\nemerged as most important for FSL performance, awarding SOTA even to the\nsimplest of techniques. These are: the backbone architecture (bigger is\nbetter), type of pre-training on the base classes (meta-training vs regular\nmulti-class, currently regular wins), quantity and diversity of the base\nclasses set (the more the merrier, resulting in richer and better adaptive\nfeatures), and the use of self-supervised tasks during pre-training (serving as\na proxy for increasing the diversity of the base set). In this paper we propose\nyet another simple technique that is important for the few shot learning\nperformance - a search for a compact feature sub-space that is discriminative\nfor a given few-shot test task. We show that the Task-Adaptive Feature\nSub-Space Learning (TAFSSL) can significantly boost the performance in FSL\nscenarios when some additional unlabeled data accompanies the novel few-shot\ntask, be it either the set of unlabeled queries (transductive FSL) or some\nadditional set of unlabeled data samples (semi-supervised FSL). Specifically,\nwe show that on the challenging miniImageNet and tieredImageNet benchmarks,\nTAFSSL can improve the current state-of-the-art in both transductive and\nsemi-supervised FSL settings by more than $5\\%$, while increasing the benefit\nof using unlabeled data in FSL to above $10\\%$ performance gain.\n