Learning to Learn with Contrastive Meta-Objective

Summary

This study presents a contrastive regularizer to improve meta-learning. Specifically, the authors propose to incorporate a contrastive meta-objective that improves the alignment and the discrimination abilities of meta-learners, leading to better task adaptation and generalization. The authors demonstrate empirical effectiveness of the proposed ConML across several meta-learning and in-context learning scenarios.

Strengths

1. The introduction of contrastive regularization sounds intuitively straightforward and motivating. 2. While equipping meta-learning with contrastive objective is not a new concept (e.g., [1], [2]) the implementation covers major meta-learning methods, including optimization- metric- and amortization-based methods. This means that the study is more comprehensive than previous studies. 3. The numerical results are promising. Code is provided - reproducibility is commendable. [1] Gondal et al. Function Contrastive Learning of Transferable Meta-Representations. In ICML 2021. [2] Mathieu et al. On Contrastive Representations of Stochastic Processes. In NeurIPS 2021.

Weaknesses

My primary concern lies in the specific contrastive strategy employed. 1. The contrastive objective aims to minimize intra-task distances while maximizing inter-task distances. However, the absence of appropriate regularization or constraints raises concerns about potential model representation collapse. This collapse could manifest as representations converging to trivial solutions, such as constant vectors or confinement to low-dimensional subspaces. The authors should address whether they have considered these risks. 2. In addition, I wonder why the contrastive objective does not follow commonly studied ones, e.g., InfoNCE, in contrastive learning. 3. Moreover, the meta-objective necessitates computations involving representations from different tasks within a batch during each episode. Since the paper lacks a discussion on training and inference efficiency, the impact of this strategy on scalability is unclear. 4. The effectiveness of this method is likely dependent on hyperparameters tuning and the sampling strategy for creating subsets of tasks. As aforementioned, incorporating contrastive learning into meta-learning is not something completely new, even though, the authors do not include detailed discussion on how different strategies would affect the performance/efficiency, which is something to be expected on my end.

Questions

please see weaknesses section. I am open to revise the score if reasonable clarifications can be provided.

Rating

4

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors do have discussed the limitation of this study.

Summary

The paper aims to enhance the meta-learning process by implementing more robust supervision within the model space. Specifically, the authors seek to augment learning capabilities through model alignment and discrimination, aiming to approximate human-like rapid learning abilities. They propose that models trained on tasks within the same super-task exhibit similarity, while those trained on different task sets generalize effectively across diverse tasks. To achieve this, the authors devise a contrastive framework that remains independent of the specific meta-learning algorithms employed. This framework encourages closer alignment of representations for models adapted to similar tasks while pushing representations apart for models derived from dissimilar tasks. Moreover, the framework seamlessly integrates with optimization-based, metric-based, and amortization-based meta-learning methods. The approach demonstrates improvements across standard benchmarks in all evaluated scenarios. Furthermore, the proposed method shows promise for integration into the in-context learning of large language models, resulting in observed enhancements.

Strengths

• The paper is well-written and easy to understand. It's reasonable to enforce the model to emulate human learning capabilities through alignment and discrimination. • The proposed contrastive learning framework is versatile and applicable to most meta-learning methods. • The proposed method consistently improves upon existing meta-learning methods across standard benchmarks.

Weaknesses

• The paper lacks validation on MetaDataset[A], which is a common large-scale dataset for few-shot learning tasks. • There is a need for sensitivity analysis on certain hyperparameters, such as λ and the choice of similarity function for contrastive learning. [A] Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples, ICLR2020.

Questions

When sampling batches of tasks at each iteration, if the same class appears in both tasks, it raises questions about how discrimination is computed between tasks.

Rating

6

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

Please see weakness.

Summary

This paper deals learning to learn (meta-learning) problem, from the perspective of exploring inner-task and intra-task relationship. Specifically, this paper proposed a Contrastive meta-objective by exploring intra- and inter-task distances and severed as an additional term for training objective (in addtion to classification loss). Experiments are conducted on both conventional few-shot image classification and in-context learning settings. Common benchmarks are used to compare with simple few-shot methods such as MAML, ProtoNet, SCNAPs. For In-context learning, simple synthetic functions are used for comparison.

Strengths

- The overall method makes sense. Designing intra- and inter-distance to explore the task-level contrastive information and further introduce this into to the meta-learning objective is reasonable. The diverse tasks naturally compose the contrastive pairs, useful for training. - The proposed method is general and can be applied on top of different few-shot classification/regression methods, such as metric-based, optimisation-based, simpleCNAPs, and in-context learning. - The performance gains over these simple baselines are significant, showing the effectiveness of the proposed method.

Weaknesses

- Technically, the proposed contrastive meta-objective is similar to the idea of supervised contrastive learning, which already provides good insights to the representation learning and deep learning community. Therefore, the proposed method is kind of incremental and provides less new knowledge to the field. - The method is only verified on top of simplest baseline methods (MAML, ProtoNet, etc). In meta-learning, various works have been proposed to investigate the possible exploration of the task-level information for improved meta-learning, such as [R1-R4], to name a few. However, none of those previous efforts were discussed or compared. Only beating the naive baseline cannot comprehensively demonstrate the advantages of this paper. - Experimentally, the proposed method tries to show its superior performance over simple baselines rather than SOTA. This is less convincing. - The in-context learning experiments are only on simple synthetic data, lack of significance. - The tile and scope: Learning-to-learn is very general, but in fact only classification related experiments are conducted. By convention, the learning-to-learn approaches will also verify on reinforcement learning. [R1] Fei, N., Lu, Z., Xiang, T., & Huang, S. (2021). MELR: Meta-learning via modeling episode-level relationships for few-shot learning. In International Conference on Learning Representations. [R2] Agarwal, P., & Singh, S. (2023). Exploring intra-task relations to improve meta-learning algorithms. arXiv preprint arXiv:2312.16612. [R3] Han, J., Cheng, B., & Lu, W. (2021). Exploring task difficulty for few-shot relation extraction. arXiv preprint arXiv:2109.05473. [R4] Zhang, Tao. "Episodic-free Task Selection for Few-shot Learning." arXiv preprint arXiv:2402.00092 (2024).

Questions

Please see above Weaknesses

Rating

3

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

N/A

Summary

The paper proposes a contrastive meta-objective that can be applied to various meta-learning methods. Also, interpreting in-context learning as a meta-learning formulation, extended the proposed method to in-context learning. Specifically, the objective is to contrast task identity obtained after episode optimization. The task identity is defined as the model weight or the feature obtained by feed-forwarding, in the case of in-context learning. Finally, the authors demonstrated the superiority of the proposed method by applying it to several meta-learning methods to improve their performance.

Strengths

- The proposed method can improve diverse meta-learning methods. - Contrasting task identity sounds intuitive.

Weaknesses

In general, the text is not easy to understand. - Not self-stained figures. - In Figure 1, it's hard to understand what $h_{w_i}$ and $w_i$ are since the caption has no explanation. - In Table 1, the caption could have included the definition of $g$ or $\psi$. - Figure 2 is hard to read; (b) and (e) missed the x, y-axis meaning and are too small to see something. - Experiment details are missing. - Hard to understand Section 5.1. Though the sine wave regression problem is well-known in this domain, it's hard to interpret results without task definitions. - The tasks in Section 5.3 are not clearly defined. - An ablation study would be helpful. - How to decide distance function $\phi$? - What if increasing the number of task-sampling $K$?

Questions

- In the case of optimization-based methods, should the method need to proceed the episode optimization twice, once for $D^{tr}\cup D^{val}$ and once for $D^{tr}$? Then why does it only take 1.1~1.5x time only? - In the case of metric-based methods, if class order is shuffled, the model presentation $[c_1|c_2|...|c_N]$ also changes? - In the case of ICL, does "consistent value in an episode" mean the same value is used for every task and contrast - Can using a feature after feed-forwarding a random input to the model be also used for the other settings, i.e. meta-learning methods? - In case of two tasks share almost the same classes (e.g. task 1 consists of class 1,2,3 and task 2 consists of class 1,2,4), should we consider these two tasks to be discriminated much? - Why "alignment" is related to fast-adaptation? Isn't fast adaptation related to the number of steps to converge, not the number of shots? - Why "discrimination" is related to the generalization?

Rating

4

Confidence

4

Soundness

2

Presentation

1

Contribution

2

Limitations

N/A

Gentle Reminder

Dear all reviewers, We appreciate your sincere and constructive feedback on our paper. During the rebuttal period, we think our paper was improved to answer all your concerns and questions. We also believe that our paper can be further advanced during this discussion period. We would appreciate it a lot if reviewers could discuss the response and re-evaluate our paper based on those discussions, and we would cherish any further comments and suggestions. Sincerely, Authors

Response to Authors

I would like to thank the authors for their time and effort in addressing my concerns. My major concerns have been resolved and the evaluation of MetaDataset has shown great improvement. After reading other reviewers’ comments and authors’ rebuttals, I decide to maintain my score and remain positive.