Information Guided Regularization for Fine-tuning Language Models

Summary

This paper examines the relationship between task-sensitive parameters and the loss landscape during pre-training. It investigates dropouts and introduces a new variant called 'guided dropout.' The experiments conducted demonstrate that the proposed guided dropout consistently yields better performance. The clarity of the writing is fair (but with some technical details missing), however execution quality, originality of the work can be improved (see reasons to reject).

Rating

5

Confidence

4

Ethics flag

1

Reasons to accept

- The author proposed an interesting training method by sampling layers for masking out. - The proposed guided dropout seems simple to implement and is effective with the BERT model. - The cross-correlation analysis on the effects of training data and sparsity for FIM calculation provides valuable insights, challenging common assumptions in this area.

Reasons to reject

- While containing interesting experiments, the connection between FIM, sparsity, and the proposed algorithm is unclear. It would be better if the author provided an algorithm block. Currently, it is unclear how Fisher information is used as part of the algorithm. - The paper ignored prior literature that studied sparsity, fisher information, and transfer learning with no related work section. This leads to potential insufficient baselines, given a few examples (all can serve as baselines in fact…): - Sung, Yi-Lin, Varun Nair and Colin Raffel. “Training Neural Networks with Fixed Sparse Masks.” – using FIM to determine a submask for training transformers, experimented with GLUE tasks and BERT as the backbone. - Liu, Chen Cecilia, Jonas Pfeiffer, Ivan Vulic and Iryna Gurevych. “FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing.” – using FIM to determine the training order of adapters for cross-lingual transfers with transformers. - Lodha, Abhilasha, Gayatri Belapurkar, Saloni Chalkapurkar, Yuanming Tao, Reshmi Ghosh, Samyadeep Basu, Dmitrii Petrov and Soundararajan Srinivasan. “On Surgical Fine-tuning for Language Encoders.” - using FIM to determine which layer to train with GLUE tasks and BERT backbone. There are also non-FIM based varieties in terms of (temporarily) omitting layers for training such as LP-FT, gradual unfreezing or Auto-RGN/Auto-SNR in surgical fine-tuning…

Questions to authors

- How are P_upper and P_lower determined, if the masking follows a schedule, what’s the typical schedule? An analysis would be nice. - Table 1: Prior work already uses a fraction of training data for estimating FIM. This experimental finding only holds when computing FIM with respect to a subset of parameters. This should be clarified further in Observation 2 in writing. While this analysis is interesting, would you please clarify its connection with the layer-wise masking dropout discussed later in the paper? - How does the proposed method work on other type of transformers (other than BERT)? ----- Updated my score to reflect the opinion change after rebuttal.

Summary

Based on the relationship between (empirical) Fisher information and (the expected) hessian matrix, as well as the established observation that "sharpness" of Hessian's correlation with generalizability, this paper proposes to measure "how well the pretrained parameters are trained" based on the scale of Fisher score, i.e., parameters with higher Fisher scores are less well pretrained. Built on this conjecture, this paper developed a variant of dropout to apply different strengths of regularization to different layers during the fine-tuning stage. Overall I find this paper interesting and potentially useful in various downstream applications.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

1. Simple yet effective method 2. Illustrative analyses 3. Efficient estimation of Fisher scores

Reasons to reject

1. I think the authors can improve their writing by offering more details. Especially, I am curious about 1) what is the actual cost of Fisher score estimation? 2) which layers are with higher and lower Fisher scores? 3) In Figure 2, we see a big difference between 40&60, what happened there? 2. Since the loss visualization is quite central to the analyses, offering a brief preliminary section would be beneficial (longer and more detailed than the part in 2.2). 3. The paper in my opinion would be strengthen by adding more experiments: 1) adding some experiments on OOD. I wonder whether keeping flatter parameters can benefit OOD generalization, and 2) (less important) adding experiments on larger decoder models. As I understand the compute requirement here, I believe adding more "modern" architectures will better show the effect of guided dropout.

Questions to authors

See the first point in reasons to reject.

Summary

This paper proposes a novel approach to regularization in the context of fine-tuning language models (LMs). By utilizing Fisher information to analyze the effects of task-sensitive parameters on the loss landscape of pretrained models, the authors introduce a method called "guided dropout." This technique adjusts dropout rates layer-wise based on the Fisher information of parameters, aiming to minimize the perturbation from optimal convergence and enhance model generalization, especially under conditions of limited data availability. Their empirical results suggest that this approach outperforms traditional regularization methods across various tasks, particularly in data-scarce scenarios.

Rating

7

Confidence

3

Ethics flag

1

Reasons to accept

1. The use of Fisher information to guide dropout regularization provides a novel and theoretically motivated method to improve LM generalization, particularly in fine-tuning phases. 2. The proposed method is versatile, applicable to various LMs (as demonstrated with BERTBASE, GPT2, and T5), and independent of the specific downstream tasks, which enhances its utility across different settings. 3. The comprehensive empirical testing, including comparisons with standard regularization methods across multiple GLUE tasks, robustly supports the effectiveness of the guided dropout approach.

Reasons to reject

1. While the results are promising, the experiments are limited to a specific subset of models and tasks. Extending these findings to a wider array of models and scenarios would be beneficial to substantiate the universality of the approach. 2. The effectiveness of the approach is heavily reliant on the accurate estimation of Fisher information, which might not always be feasible or reliable, especially with very large models or complex datasets. 3. Despite claims of architecture agnosticism, the actual implementation and effectiveness may vary significantly across different architectures, which was not fully explored beyond a few model types.

Summary

The paper performs extensive empirical studies on the relationships between Fisher information and loss landscape during LM training. They propose an adaptive dropout technique based on their empirical findings, which has demonstrated to be effective across multiple downstream tasks and multiple types of LMs.

Rating

6

Confidence

4

Ethics flag

1

Reasons to accept

The results are very interesting to know. So I think it will be good for the community to see this paper. The 4 key observations are very interesting and I enjoy reading about them. The empirical results verify their hypothesis and demonstrate their proposed methods. The experiments are solid.

Reasons to reject

The experiments are somewhat limited. BERT, T5, and GPT2 are very interesting models, but they are somewhat obsolete. The authors claim that "Thus, we believe our regularizer to be universally applicable." But this seems to be over-claim because there are no experiments---please correct me if I am wrong---about new generation of LMs like Llama or Mistral (even the smallest versions).

Questions to authors

What do you think of its potential generalization to larger decoder-only models? What do you think of its potential generalization to other architectures, like SSM?

Re authors

Thank you very much for the additional information. *Their Problem vs Our Problem*: their problem don't perform architectural change... Sung et al. can be seen as parameter-wise dropout using FIM. Liu et al. is 2023, not 2024 --> can be seen as layer-wise progressive dropout using FIM RGN/SNR (Lee et al, 2022 from surgical FT) --> archiving the similar effect by reduce the update of parameters that get's small update --> parameter-wise - It is interesting and important to compare the strength of layer-wise versus parameter-wise method especially FIM are estimated. - Thank you very much for the additional experiment using Lodha 2023.