Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks

In this paper, we study the generalization properties of Model-Agnostic\nMeta-Learning (MAML) algorithms for supervised learning problems. We focus on\nthe setting in which we train the MAML model over $m$ tasks, each with $n$ data\npoints, and characterize its generalization error from two points of view:\nFirst, we assume the new task at test time is one of the training tasks, and we\nshow that, for strongly convex objective functions, the expected excess\npopulation loss is bounded by ${\\mathcal{O}}(1/mn)$. Second, we consider the\nMAML algorithm's generalization to an unseen task and show that the resulting\ngeneralization error depends on the total variation distance between the\nunderlying distributions of the new task and the tasks observed during the\ntraining process. Our proof techniques rely on the connections between\nalgorithmic stability and generalization bounds of algorithms. In particular,\nwe propose a new definition of stability for meta-learning algorithms, which\nallows us to capture the role of both the number of tasks $m$ and number of\nsamples per task $n$ on the generalization error of MAML.\n