L2T-DLN: Learning to Teach with Dynamic Loss Network

Summary

The paper introduces the concept of teaching in machine learning and proposes a framework called L2T-DLN (Learning to Teach with Dynamic Loss Network). The framework aims to address the limitations of existing approaches by incorporating the temporal nature of loss function adjustment and utilizing the states of the loss function. The authors formulate the loss adjustment as a temporal task using a teacher model with memory units and propose a dynamic loss network to enhance the interactions between the teacher and student model. Extensive experiments demonstrate the effectiveness of the approach on various real-world tasks.

Strengths

1. The paper presents a novel framework, L2T-DLN, which integrates the concept of teaching with dynamic loss functions. This approach stands out by considering the temporal nature of loss function adjustment and utilizing the states of the loss function, setting it apart from existing works. 2. The authors provide a thorough convergence analysis of the proposed framework, demonstrating its effectiveness and its ability to achieve convergence.

Weaknesses

1. Unclear Motivation: The paper lacks a thorough discussion of the benefits of introducing the gradients concerning loss functions. It would be beneficial to include a clearer motivation. 2. Marginal Improvement: The reported improvements of 0.4% lower than ALA on ImageNet for classification and only 0.3% improvement compared to the baseline on mIoU on the VOC dataset for segmentation are relatively small.

Questions

1. Further clarification is needed in the paper on why it is necessary to consider the gradients concerning loss functions as a core idea. Besides experimental results, could there be theoretical analysis supporting this aspect? 2. The authors introduce a Dynamic Loss Network instead of using a dynamic loss function. It would be helpful to explain if this choice leads to a larger number of parameters and potential unfairness in comparisons. Please clarify the impact of parameter quantity in this context.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

The implementation of this approach necessitates substantial computational resources for calculating high-order derivatives.

Summary

This paper introduces an improvement towards the learning-to-teach framework. Compared with the previous works, the authors made an innovation that a dynamic loss network is added with LSTM acting as teacher model to enhance the temporal memorization. A three step optimization procedure is also proposed with an convergence analysis theoretically. Last extensive experiments are conducted on various computer vision tasks that demonstrated the effectiveness of the proposed approach.

Strengths

1. The problem this paper studied is an important one yet insufficiently studied in the current literature, that is how to use a better teaching strategy in addition to merely conduct "learning". It's a very meaningful step forward of this paper that pushes forward the frontier of such a literature; Furthermore the design of dynamic loss network is a new perspective for this area as well, going beyond the dynamic loss function design. 2. The paper is technically very sound with enough insights and depth. I particularly appreciate the convergence analysis which is done first time for the machine teaching/learning-to-teach area, with a good depth leveraging the negative curvature.

Weaknesses

As also stated in the paper, despite being theoretically sound and elegant, the proposed algorithm incurs much more computational overhead which will hinder its practical usefulness in real world applications. Particularly the compute of Hessian even three-order derivatives is a sign of its computational complexity. I agree with the author that this needs to be improved further in the future to enlarge the impact of this work.

Questions

While the memory units of LSTM make sense in this scenario, how about use Transformer as another sequential model which may even enhance the temporal memorization compared with LSTM?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

Stated in the paper as well as the above point, that the computational complexity is one concern while it seems feasible and justifiable to further solve this, as the future potential value of this direction is likely large.

Summary

In this paper, the authors state that existing methods only employ a simple feedforward network as the teacher model, which limits the potential of L2T. This issue motivates authors to propose a network with a memory unit to enhance the temporal analyzing ability of the teacher in the learning to teach (L2T) tasks. This proposed new network is combined with a dynamic loss network to address the issue of the existing works above. The experiment shows the superiority of the proposed method.

Strengths

(+) This paper theoretically analyzes the convergence of the proposed method. (+) The proposed method shows the state-of-the-art performance compared with other methods.

Weaknesses

(-) The novelty of this paper is not clear. In the title, abstract and Introduction, the authors repeatedly emphasize that they propose an L2T framework with a Dynamic Loss Network (L2T-DLN). However, these emphases can cause misleading since Dynamic Loss has already been introduced in previous work, as stated by the authors themselves in Lines 2, 23, and the caption of Figure 1. Consequently, the application of Dynamic Loss in L2T cannot be considered as the novelty or the main contribution of the paper. The continued highlighting of DLN in these sections adds to the confusion and may hinder the understanding of the paper. In my understanding, the primary contribution of this paper is a proposed network with a memory unit to enhance the temporal analyzing ability of the teacher in the learning to teach (L2T) tasks. By emphasizing this aspect, rather than the Dynamic Loss network, the paper will become more coherent and easier to comprehend. (-) The authors have not adequately demonstrated the effectiveness of their proposed method in both theory and practice. In the theoretical analysis, they solely focus on the convergence of the proposed L2T-DLN and fail to illustrate how the proposed network with a memory unit for the teacher can benefit the training of L2T task. Additionally, the experiments show the better performance of the proposed method compared with other SOTA methods. However, based on section 5.1, this better performance is based on many combinations of loss functions and tricks. It remains unclear whether the proposed network with a memory unit for the teacher contributes to the enhanced performance observed in the L2T task. (-) The paper lacks sufficient ablation studies to support its claims. In the ablation study section, the authors include some experiments related to selecting the best learning parameters for their proposed method; these parameters may primarily influence performance rather than directly relating to the main contributions of the proposed method. Therefore, it is crucial for the authors to conduct a series of ablation studies to demonstrate how the proposed network with a memory unit for the teacher can benefit the training of L2T tasks.

Questions

See weakness.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

1 poor

Contribution

1 poor

Limitations

No limitations

Summary

This paper presents a three-stage framework that dynamically adjusts the learning process of student model (i.e., target model). The loss value is calculated via the proposed Dynamic Loss Network (DLN), parameterized as neural network. And the DLN is updated by the teacher network implemented as LSTM. The authors also provide a convergence analysis. The model is evaluated on an array of tasks including image classification, object detection and semantic segmentation.

Strengths

1. Compared to previous works, L2T-DLN leverages the state of loss function and the temporal information of student learning experience to update the teacher. 2. Experiments on various application including image classification, object detection and semantic segmentation showcase the effectiveness. 3. The provided convergence analysis is helpful to understand the framework.

Weaknesses

1. The proposed DLN is not unified. For image classification, it produces the loss value. For object detection, it generates the loss weights. 2. Some technical details of the DLN are missed, as will discussed in next session. 3. To access the temporal information of student learning, L2T-DLN would store the historical models and consume lots of memory. 4. Since the DLN adjusts the loss weights of the objectives of YOLO-v3, it’s reminiscent of multi-task learning. In this regard, the authors should discuss the relationship with MTL.

Questions

1. The output layer (e.g., activation function) of the DLN is not clearly described. 2. It would be better to understand the DLN if the loss value/weights can be visualized. 3. Figure 1 in the supplementary material indicates the DLN (blue line) would produces lower gradients applied on the student model. Is this because the baseline model uses a large learning rate (0.1 line 219)? 4. The claimed effectiveness would be more convincing if an ablation study of learning rates is provided. 5. Regarding the initial gradients produced by the DLN, which greatly exceeds the baseline, it may be related to the network initialization of the DLN. Since this detail is missing, I would suggest the author clarifying the initialization. 6. For the DLN of object detection, is it initialized as identical mapping? If not, it’s helpful to investigate this manner. 7. What’s the training epoch for baseline model and the L2T-DLN for image classification?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

1. Some details (e.g., initialization and activation of the output layer) of the proposed DLN are missing. 2. The used learning rate for image classification may not be optimal, an ablation study would help understand the framework. 3. It would be helpful to visualize the output of DLN. 4. The framework may consume lots of GPU memory.