Post-Hoc Reversal: Are We Selecting Models Prematurely?

Summary

This paper shows empirical evidence that common approaches to model selection can be improved upon by less greedy alternatives. In particular, authors highlighted what they referred to as post-hoc reversal, when post-training transformations reverse the trends observed in independent training runs. For instance, in certain overfitting situations, transformations such as temperature scaling, weight averaging, and ensembling, reversed growing test error trends and closed the generalization gap to a greater extent than common model selection approaches. This observation then yielded practical recommendations of incorporating post-training transforms into the model selection pipeline, which resulted in improved performance across a number of settings, as well as on smoother error curves that render model selection less noisy. Authors attributed post-hoc reversal to the high variance involved in the training of neural networks, which can be smoothed out by some of the studied transformations. The paper lays out clear practical recommendations as a result of the reported empirical observations, which I believe would be highly relevant to the community.

Strengths

- The paper is well written and easy to follow, and tackles a highly relevant problem in practice. - The evaluation is broad and covers a number of model classes and sizes, and data modalities. - Conclusions lay out very clear and easy to implement recommendations to improve model selection.

Weaknesses

- This requires no action from the authors but just for the record, I have mixed feelings about the presentation choices regarding the use of notation. The text is notation heavy, which makes it precise, but at the cost of readability. The discussion seems simple enough to enable the use of almost plain text only. But I reiterate that I consider this to be mostly matter of writing style, and I expect no effort from authors addressing this comment. - The bulk of the empirical assessment focuses on a somewhat small scale setting involving variations of the Cifar-10 dataset and relatively small neural networks. While this is obviously due to the high cost involved in replicating these experiments in larger scale settings, it limits the strength of the presented empirical evidence. However, section 6 adds results for other settings including fine-tuning of very large models, which address this concern to an extent. - The experiments focus on multi-epoch training settings, and it's a bit unclear how those results transfer to now common pre-training situations where very few epochs are used. While one can replace epoch-end checkpoints by checkpoints obtained every k steps, it's unclear how the choice of k affects results, for instance. - One component that seems to have been left out of the analysis is robustness. Does post-hoc reversal still happens in situations where the test set is somehow shifted relative to the validation set used for post-hoc selection? It might be the case that there's not so pronounced of a gap between post-hoc and greedy/naive selection in such a scenario.

Questions

- Would it be possible to replicate a subset of the results reported in section 4 and 5 for a larger scale dataset (e.g., ImageNet)? - Does post-hoc reversal still happen in situations where the test set is somehow shifted relative to the validation set used for post-hoc selection? - How does one leverage post-hoc selection in a non-multi-epoch training situation?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Please refer to the "Weaknesses" section.

Summary

The authors investigate applying three post hoc transforms, namely temperature scaling (TS), ensembling, and stochastic weight averaging (SWA), to trained models after training, separately and jointly. They oppose their approach to the "naive approach" and provide empirical observations of a phenomenon they refer to as "post hoc reversal."  This phenomenon corresponds to the performance trends of models that change after these transformations. In particular, they focus on the observation in noisy settings of the following observations. 1- Epoch-wise post hoc reversal. The authors show that post hoc transforms reduce overfitting, double descent, and loss-error mismatch. In particular, SWA and ensembling reduce overfitting and flatten double descent peaks. 2- Model-wise post hoc reversal. The model's size influences the improvement in the performance of the post hoc transforms, i.e., larger models tend to perform better under post hoc transforms. 3- Hyperparameter-Wise Post Hoc Reversal. It appears that constant learning rates improve performance better than decaying learning ones and the optimal number of epoch shifts.  The experiments are broad, and metrics show good non-trivial improvement based on the method.

Strengths

1 - The paper provides an extensive empirical study across various domains (vision, language, tabular, graph), aiming to demonstrate the generality of their approach, i.e., post hoc transforms.  2- The authors use a variety of performance metrics (error rates, loss, MMLU accuracy) to validate the effectiveness of post hoc transforms, leading to a robust evaluation. 3- The authors identify a potentially impactful post hoc reversal phenomenon that could challenge commonly adopted practices. 4- The approach seems to lead to consistent performance gains across different datasets and settings. The focus is on noisy environments, which can be particularly relevant for less curated datasets, which might be what the field needs.  5- The authors suggest that their approach could lead to guiding principles for model development, including early stopping and checkpoint selection.

Weaknesses

The main weakness of the approach, which is recurrent in most current empirical LLM observations, is that understanding the source of the improvement needs to be clarified.  In fact, it is not clear why the benefits of post hoc transforms are more pronounced in high-noise settings. Making assumptions to verify potentially synthetic and controlled datasets could be helpful. As of one, this corresponds to an observation.

Questions

If the author can provide a good answer to the question above that would address my concerns.

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes

Summary

The paper discusses the phenomenon of *post-hoc reversal*, where the performance trend is reversed after applying post hoc transforms, namely, temperature scaling (TS), stochastic weight averaging (SWA), and ensembling (Ens). The paper conducts an empirical study to observe the phenomenon across different epochs, model sizes, and hyperparameters. They propose a *post-hoc selection* strategy where the optimal epoch/model size/hyperparameters are selected considering the performance after applying post-hoc transforms. The paper focuses on the noisy data setting and shows experimental gains in that setting for some datasets. Experiments have been conducted for the FMoW dataset, CIFAR-10-N, and CIFAR-100-N, as well as some additional experiments on text, tabular, and graph datasets.

Strengths

1. The paper demonstrates the phenomenon of post-hoc reversal. 2. They conduct experiments on diverse domains. 3. Performance gains are achieved with the proposed strategy in some of the noisy data settings.

Weaknesses

1. The paper proposes *post-hoc selection* strategy, simply selecting the optimal epoch (or model size, etc.) after applying already existing (and widely used) post-hoc transforms (SWA, En, TS). They do not propose any *novel* mechanism to tackle *post-hoc reversal*. 2. It is a findings paper; the study is empirical and lacks theoretical insights. Given the empirical nature of the study, the performance gains in some datasets are marginal (e.g., C-100-N Noisy test error, Table 1). 3. For the datasets in Table 5, the post-hoc selection strategy does not consistently provide performance gains, i.e., it performs worse in some cases.

Questions

**Questions** 1. Why is post-hoc reversal prominent in the noisy data setting? Can the authors provide intuition or reasoning (other than empirical observations)? 2. Can the authors provide intuitions to (at least) some of the observations? For example, why do SWA and Ens handle double descent, but TS causes it? 3. In Figure 7, what is the SWA ensemble? It has not been discussed in Section 5. 4. For the datasets Yelp, Income, and Reddit-12K, the post-hoc reversal is observed in Figure 7, but the post-hoc selection strategy either shows worse performance or marginally better. Does this indicate that the post-hoc selection strategy is ineffective in these settings? Can the authors provide any reasoning/explanations? **Suggestions** * The paper title should include "noisy data setting", as the paper focuses on the noisy data setting. * Include intuitions/reasonings/explanations (wherever possible) for the observations in the experimental results.

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

Yes.

Acknowledgement to Author's Rebuttal

Thanks to the authors for the detailed response. Most of my queries have been addressed. I do not have any further questions at this point.