A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm

Summary

The authors address an existing gap in fast adaptation of meta-learning by developing a robustification framework that draws from the concept of expected tail risk and recasts robustification as a distributional optimization problem. They adopt a heuristic two-stage optimization strategy that is broadly useful toward most meta-learning methods. The applicability of this framework toward enhancing robustness against task distributions and mitigating worst-case performance is verified and discussed over a selection of benchmark tasks.

Strengths

+ The paper motivates the importance of expanding work in distributionally robust optimization and presents a unique formulation of the robustness problem for meta learning. + Concrete illustration of how the proposed optimization method can benefit existing meta learning work. + Well-organized sections and visual structure. + Nice FAQ — the authors should consider making their contributions clearer in the main text of the paper.

Weaknesses

- Appendix J.1 states: "The optimal selection of the confidence level α is difficult since we need to trade off the worst and average performance. Our setup is to minimize CVaRα, which already considers the worst-case at some degree, so we watch the average performance in meta training results and set α = 0.7 for all few-shot regression tasks." This isn't really tail risk (e.g., alpha > 0.9, alpha < 0.05). Given the tail risk problem, why does alpha range from 0.2 to 0.8 (Figure 5)? - The paper seems incremental to TR-MAML. It would have been better to see more detail in the main text (as opposed to Appendices) on the main contribution of taking a probabilistic approach to meta learning and using a cVaR probabilistic measure (e.g., applicability, convergence, bounds). Instead, the main text focused on implementing DR-MAML which may not work for true tail scenarios. - The authors should introduce at least a new dataset versus just reusing MAML/TR-MAML. Appendix J shows that the authors simply modified the loss function, reusing code from MAML and TR-MAML. It would be nice to see additional experiments in more challenging settings, e.g., RL tasks like point navigation or continual learning. - Appendix K states: "Note that when α → 1.0, the optimization objective approaches the worst-case optimization objective. Here we attach two possible reasons for the performance degradation phenomenon:..." This should be expanded and included in the main text as opposed to the Appendix. - It seems the main contribution is imposing a probabilistic constraint (cVaR) versus using worst-case TR-MAML, which necessarily only works for meta-learning over a large task space.

Questions

- What is the justification for deriving results over only 3 runs? This makes empirical claims seem suspect, e.g. the conclusion that “DR-MAML surpasses MAML in terms of average performance” - It would be nice to see additional discussion of the impacts of assumptions 1/2/3. (The authors briefly touch on this around line 294.) - Pseudocode is in Appendix B but it would be more useful in the main text. * Experimental results for the proposed approach could be further substantiated by including a discussion of Tables 4/5 in the main text. * Minor: * line 34: “plaint” -> “plain” (note: this may be ignorance on my part.) * line 199: “sometime” -> “sometimes” * line 208: “a” -> “an” * Figure 4 caption line 2: “DR-MAMAL” -> “DR-MAML” * Appendix, line 859: “Risk-Rweighted” -> “Risk-Reweighted”

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

* The authors name the primary theoretical and practical limitations of their work. * Societal impacts are not directly applicable. The authors touch upon limitations of existing meta-learning methods for high-risk real-world scenarios, which is important.

Summary

The paper proposes a novel expected tail risk objective for meta-learning, which provides an intermediate of the worst-case and average risk minimization algorithms. The proposed algorithm can be adapted to existing meta-learning algorithms with improved robustness.

Strengths

* The paper is well-written and easy to follow. * Considering expected tail risk in meta-learning is novel to me. Controlling the worst task performance at a certain probabilistic level can improve the robustness. * The experimental results show the effectiveness of the proposed approach.

Weaknesses

* The meta risk function $\ell(D_{\tau}^T, D_{\tau}^C;\nu)$ is an empirical risk estimation that depends on the number of data points $m$ of task $\tau$. It’s preferable to consider its generalization as in [1,2,3]. * Why only adopt eight meta-training tasks and four meta-testing tasks for the experiment of mini-ImageNet in Table.3? >[1] Giulia Denevi, Carlo Ciliberto, Riccardo Grazzi, and Massimiliano Pontil. Learning-to-learn stochastic gradient descent with biased regularization. In International Conference on Machine Learning, pages 1566–1575. PMLR, 2019a. >[2] Qi Chen, Changjian Shui, and Mario Marchand. Generalization bounds for meta-learning: An information-theoretic analysis. Advances in Neural Information Processing Systems, 34: 25878–25890, 2021. >[3] Yu Bai, Minshuo Chen, Pan Zhou, Tuo Zhao, Jason Lee, Sham Kakade, Huan Wang, and Caiming Xiong. How important is the train-validation split in meta-learning? In International Conference on Machine Learning, pages 543–553. PMLR, 2021.

Questions

See the discussion in the weaknesses section.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

Yes.

Summary

This paper considered a robust meta-learning framework by addressing the worst-tasks performance. Specifically, this paper adopted conditional value at risk (CVR) to filter poorly performed tasks. In order to effectively learn the CVR, this paper adopted a surrogate loss, a variant of hinge loss with a learnable threshold. The proposed framework exhibits empirical improvement in certain few-shot tasks.

Strengths

In general, this paper tackles an interesting problem in mete-learning. I would think the following points might be interesting. - Addressing robustness issue in meta-learning. - A principled and simple method through the tool of conditional value at risk. - The paper is quite clearly written.

Weaknesses

This reviewer found there are several concerns in the current version. The factors that make this reviewer suggest that a major revision is better. I will discuss these points in the following aspects. **Originality** and **Significance** The main motivation of the proposed method is to address the robustness in meta-learning. However, this reviewer feels a bit confused about its motivation in choosing CVR based approaches. There are alternative methods in distribution robust optimization such as group dro, which can be naturally extended by adding the task weights as a learnable parameter. Or if we adopt Wasserstein distance, we could generate new samples as a data augmentation training. Based on these, I would think the proposed method may lack sufficient motivation or comparison to justify its benefits. **Quality** I have some technical concerns. - In equation (2), the worst performing group sometimes is quite risky since it would have quite arbitrary value. It should be better to consider the robustness in the context of distributional robustness optimization. - In sec 4.3, I would think there are better principles to learn the threshold value. I mean, since the learnable thresholding value is convex for the loss in (6), thus we could directly adopt several solvers to solve the optimal value after each epoch. This could construct a principled method to optimize the hyper-parameter. - In the few-shot context, the worst-performing groups in general have quite few data, does the accuracy really make some statistical sense? - The representation learning based meta-method seems not empirically validated? **Clarity**. Most parts within the paper are clear. Some parts are still a bit confusing, for example - Sec 4.1 128-129 For any \theta, the meta learning.. It should be for any **fixed** \theta, right? - The surrogate seems quite similar to the hinge loss, right? I would think a further discussion is better. - DN-CRP is better named as meta-representation learning. - Fig 2 seems a bit confusing because of \alpah_{t}, actually \alpha here is fixed. The threshold value was changing w.r.t. t rather than \alpha.

Questions

In general I would think this paper has several merits. However, this reviewer has certain concerns on the novelties and technical parts. Thus currently I vote for a weak rejection. Authors are suggested to check weakness points to know details.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

N/A

Summary

In this paper, the authors present a method by which to improve the expected tail risk during meta learning via a two-stage optimisation strategy. In addition, the authors provide theoretical and empirical analysis of their proposed approach, demonstrating improvements in the average, worst-case and conditional value at risk (CVar) metrics.

Strengths

The paper is written and structured very well. In particular, Sections 2,3, and 4 provide great introductions to the meta-learning paradigm and the idea of risk-minimisation applied to meta learning. The theoretical analysis outlined in Section 4 is also quite clear. Looking at the supplementary material, proofs and assumptions are outlined well, and ablations are provided.

Weaknesses

- This paper is quite strong, and many of my initial concerns are answered in the supplementary material. - Besides minor issues with regards to wording and grammar, this paper is quite well written. For example, on lines 34-35, 'suffering from tribulation' and 'plaint'. - A minor concern, but one which might help practitioners. Choosing the optimal meta-batch size appears quite important to the final performance. Although the authors discuss the influence of task batch size, perhaps ablating across more (and a wider range of) batch sizes might be more informative. For other weaknesses/comments, see questions.

Questions

- In the paper, the Monte Carlo estimate allows for the theoretical analysis, but it is indeed quite crude. Do the authors have any comments on perhaps improving upon this (even if the theory isn't there yet)? - What optimizers did the authors use? In the existing limitations section, they suggest that the batch size selection has a greater effect for first-order optimizers. Did the authors attempt to use any higher order (or approximate) optimizers? - I believe the theoretical analysis breaks down for the image classification tasks? (please do correct me if I'm wrong) Do the authors have any further insights on this point, in particular as to why the tail risk is not well optimised for omniglot, but works well for mini-ImageNet? - The authors also state that challenging meta-learning tasks can reveal more advantages of DR-MAML (line 281). Have more challenging tasks been tried? This could, for example be the meta-dataset or something equivalent? - In Figure 5, there appears to be deviations from the trend when $\alpha = 0.7$, and in general, the trend appears to be somewhat non-monotonic. Is this a dataset dependant issue? - Do the authors have any general guidelines for how the image classification performance of DR-MAML changes with $\alpha$?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The limitations of this work are outlined in Section 6 and assumptions are outlined throughout the paper. In addition, error bars and std's are provided for all experiments. Overall, I see no potential negative societal impacts of this work. If anything, robust meta-learning has a potentially positive societal impact for deployed meta-learning systems.

Summary

This paper proposes a novel meta-learning pipeline from the perspective of distributionally robust perspective to robustify fast adaptation. Through a two-stage strategy heuristically, the model could control the worst fast adaptation cases at a certain probabilistic level. And experiments show the effectiveness of the proposed model compared to other baselines, especially TR-MAML.

Strengths

1. The proposed idea is interesting, and it could generalize the expected risk minimization and worst-case risk minimization for meta learning together. 2. The presentation and structure of the paper is good. 3. The experiments are extensive for the proposed model DR-MAML.

Weaknesses

1. It is a difficult to follow for section 4. Some motivations or intuitions are necessary. For instance: - what is the goal to introduce $(R^{+}, B)$ above line 129? It does not appear in the following paper. - what is the intuition to involve the slack variable $\xi$ below line 157? 2. Some notations are not clear. For instance: - line 134: why $M^{-1}(\ell)$ instead of $M(\ell)$? - For Eq.(1) and (3), is it better to add "min" after the ":="? - line 208: "a optimization..." -> "an opt..." 3. Do you consider diverse tasks during meta-training or just follow the common setting used in MAML? Since the goal is to robustify the fast adaptation, if you just use the common setting in MAML, how could you validate whether your model is helpful to keep robustness? Or the common setting is not robust? 4. Could you add some experiments for Example 2: DR-CNP?

Questions

see weakness.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

yes

Thanks

Thanks very much for the clarification and responses, and apologies for the misconception of CVaR on my part. The edits and new additions introduced are helpful to boost the paper, and my concerns have been addressed. I will raise my score accordingly.

Rebuttal Response

Dear Authors, Thank you for your detailed response. 1. Using a deep density estimation (something like: https://arxiv.org/abs/2107.11085) approach for VaR is interesting. I wonder if this might help or hinder the memory issues? 2. Additional experiments are always welcome. 3. With the previous and additional experiments, I wonder what performance benefits could be achieved with better (more stable, higher order etc) optimisers, rather than taking the ones used for TR-MAML? 4. I agree that moving beyond Omniglot is a good idea. Usually when benchmark performance goes beyond 90%, they become far less illustrative of the differences between methods. These points are perhaps something for future work, but this discussion is interesting nonetheless. With the impact/scope of this paper, I believe my initial score is correct, especially given the new results. If the authors disagree or have any additional comments, please feel free to comment below.