Bridging Few-Shot Learning and Adaptation: New Challenges of Support-Query Shift

Few-Shot Learning (FSL) algorithms have made substantial progress in learning\nnovel concepts with just a handful of labelled data. To classify query\ninstances from novel classes encountered at test-time, they only require a\nsupport set composed of a few labelled samples. FSL benchmarks commonly assume\nthat those queries come from the same distribution as instances in the support\nset. However, in a realistic set-ting, data distribution is plausibly subject\nto change, a situation referred to as Distribution Shift (DS). The present work\naddresses the new and challenging problem of Few-Shot Learning under\nSupport/Query Shift (FSQS) i.e., when support and query instances are sampled\nfrom related but different distributions. Our contributions are the following.\nFirst, we release a testbed for FSQS, including datasets, relevant baselines\nand a protocol for a rigorous and reproducible evaluation. Second, we observe\nthat well-established FSL algorithms unsurprisingly suffer from a considerable\ndrop in accuracy when facing FSQS, stressing the significance of our study.\nFinally, we show that transductive algorithms can limit the inopportune effect\nof DS. In particular, we study both the role of Batch-Normalization and Optimal\nTransport (OT) in aligning distributions, bridging Unsupervised Domain\nAdaptation with FSL. This results in a new method that efficiently combines OT\nwith the celebrated Prototypical Networks. We bring compelling experiments\ndemonstrating the advantage of our method. Our work opens an exciting line of\nresearch by providing a testbed and strong baselines. Our code is available at\nhttps://github.com/ebennequin/meta-domain-shift.\n