Summary
The main contributions of this paper are threefold:
- The authors focus on analyzing the popular meta-training procedures with two consecutive learning phases, i.e., the conventional pretraining and the episodic training.
- Based on the inefficient learning behavior during the two-phase training, which implies the probable conflicts between the objectives of the two phases, the authors combine them in a single training phase by designing a bi-level optimization process comprising two objectives for the conventional classification training and the few-shot training.
- The authors have confirmed that the proposed training called Boost-MT shows performance gains over the prior methods.
Strengths
**Strength 1:** To the best of my knowledge, the efforts to scrutinize the learning behaviors of the conventional two-phase training for meta-learning have remained unexplored. Although the two-phase training is widely used as a routine process, not much is known about the reason why the process shows performance gains beyond the single-phase episodic training. Also, we do not know much about the optimality of the two-phase process in training deep models with strong generalization. This work has found that the two-phase training shows somewhat conflicting behaviors between phases (as shown in Fig. 1), which makes re-design the learning procedure of few-shot learners. That is the most interesting and valuable contribution of this paper.
**Strength 2:** The proposed single-phase training process is fully model-agnostic, and we can even say that it is algorithm-agnostic. Therefore, I expect the proposed method can be widely applied to the prior meta-learners and other possible future learners.
Weaknesses
**Weakness 1:** The main concern is that the paper does not explicitly verify the synergetic effects between the inner (i.e., training of classifiers via a conventional full-class classification task) and outer (i.e., episodic training with a few samples) optimizations. In my opinion, the most important and fundamental question is the synergetic effect between conventional training and episodic training. Some authors have tried to focus on the differences between conventionally-trained and meta-trained feature extractors (Goldblum et al., 2020), but not many facts are unexplored yet. Therefore, I expect this work to unveil the role of the inner and outer optimization of the proposed method to fully understand the effects of the full-class classification objective and the few-shot training objective, respectively.
**Weakness 2:** More experiments for validating the applicability of the proposed method would be required. I believe that one of the biggest advantages of this method is its wide applicability. Because this work touches on the training procedure, which is algorithm-agnostic, potential gains from applying the training procedure to a wide range of meta-learners would clearly elucidate the novelty of this method. However, only a naive version of 'Boost MT', which applies the training procedure to a simple ProtoNet style model, is experimented with. Will the method yield consistent gains when applied to other early meta-learners, including TPN (Liu et al., 2019) and TADAM (Oreshkin et al., 2018), or relatively recent approaches, including CAN (Hou et al., 2019) and STANet (Lai et al., 2023)? The first group of prior works is advanced metric-based meta-learners, and the second group is based on attention or transformer to boost the few-shot learning performance, which is now positioned as the state-of-the-art.
**Weakness 3:** Further ablation studies that verify the algorithmic procedure in the proposed training should be provided. To the best of my understanding, I cannot find any ablation studies that answer the question of why we have to devise training procedures with the inner and outer structures. Also, what is the reason behind the losses for the inner and outer optimizations? What happens when we drop one of the inner and outer optimizations? In fact, these questions are closely related to 'Weakness 1' for verifying the synergetic advantages of the proposed training procedures.
There are some minor errors:
- In Abstract, it would be better to say "Also," rather than "And" at the beginning of a sentence.
- In Introduction, please add a citation of 'Meta-Baseline', which is directly analyzed in the motivating experiment shown in Fig. 1.
M. Goldblum et al. "Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks," ICML 2020.
Y. Liu et al., "LEARNING TO PROPAGATE LABELS: TRANSDUCTIVE PROPAGATION NETWORK FOR FEW-SHOT LEARNING," ICLR 2019
R. Hou et al., "Cross Attention Network for Few-shot Classification," NeurIPS 2019.
J. Lai et al., "SpatialFormer: Semantic and Target Aware Attentions for Few-Shot Learning," AAAI 2023.
Questions
**Q1:** Would you provide an in-depth explanation of the role of the inner and outer optimizations of the proposed method?
**Q2:** What are the synergetic effects between the inner and outer optimizations? Are there any empirical results or conceptual insights that explain the particular training procedure?
**Q3:** When combined with further complicated meta-learners (rather than a simple ProtoNet style version in the reported results), will Boost MT yield consistent gains? Specifically, the prior works aforementioned in Weakness 2 can be additional baselines, or other prior algorithms that are already compared in Tables 1 and 2 can be a baseline to apply Boost MT. When specifying the question, what is the accuracy of 'Boost MT + Method' beyond 'Method'?
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.