The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning

Biased regularization and fine-tuning are two recent meta-learning\napproaches. They have been shown to be effective to tackle distributions of\ntasks, in which the tasks' target vectors are all close to a common\nmeta-parameter vector. However, these methods may perform poorly on\nheterogeneous environments of tasks, where the complexity of the tasks'\ndistribution cannot be captured by a single meta-parameter vector. We address\nthis limitation by conditional meta-learning, inferring a conditioning function\nmapping task's side information into a meta-parameter vector that is\nappropriate for that task at hand. We characterize properties of the\nenvironment under which the conditional approach brings a substantial advantage\nover standard meta-learning and we highlight examples of environments, such as\nthose with multiple clusters, satisfying these properties. We then propose a\nconvex meta-algorithm providing a comparable advantage also in practice.\nNumerical experiments confirm our theoretical findings.\n