MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts

Summary

The paper addresses the challenge of estimating the test accuracy of pre-trained neural networks on out-of-distribution (OOD) samples without access to ground-truth labels. Current logit-based methods often suffer from overconfidence, leading to prediction bias. The authors propose a new method called MANO, which applies data-dependent normalization on the logits and uses the Lp norm of the normalized logits matrix as the estimation score. This approach aims to reduce prediction bias and improve accuracy estimation. Theoretical and empirical analyses demonstrate that MANO outperforms existing methods across various types of distribution shifts and architectures.

Strengths

1. The paper provides a solid theoretical analysis connecting logits to model uncertainty and generalization performance, supported by the low-density separation assumption. 2. The paper is well-written and logically structured, making it easy to follow the authors' arguments and methodology. 3. Extensive empirical studies on multiple benchmarks show that MANO outperforms existing state-of-the-art methods in different distribution shift scenarios.

Weaknesses

1. How can you ensure the universality of the Low-Density Separation (LDS) assumption in practical applications? Are there specific scenarios or datasets where this assumption might not hold? 2. What are the specific mathematical advantages of the SoftTrun normalization strategy compared to traditional softmax? To what extent do these advantages rely on particular data distributions? 3. In Table 1 and Table 2, the performance improvements over existing methods appear to be relatively minor in many instances. Is it possible to calculate other uncertainty measures to assess the significance of these results more accurately? 4. In most of the experimental results, the outcomes are close to the maximum value of the evaluation metrics. Does this imply that the research problem has already been solved?

Questions

See weakness.

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

N/A.

Summary

​​This paper presents MANO, a straightforward and efficient training-free approach for estimating test accuracy in an unsupervised manner, leveraging the Matrix Norm of neural network predictions on test data. The method is inspired by the low-density separation assumption, which posits that optimal decision boundaries should reside in low-density regions. An extensive empirical study on standard unsupervised accuracy estimation benchmarks reveals that MANO consistently achieves state-of-the-art performance across diverse architectures, even in the presence of synthetic, natural, or subpopulation shifts.

Strengths

1. It demonstrates that logits can effectively indicate generalization performance by reflecting distances to decision boundaries, in alignment with the low-density separation assumption. 2. It introduces MANO, a training-free method for estimating test accuracy by computing the $L_p$ norm of the logits matrix, which quantifies global distances to decision boundaries. MANO employs a novel normalization technique that balances information completeness and error accumulation, and is resilient to various calibration scenarios. Additionally, it reveals a connection to the model’s uncertainty. 3. It conducts a comprehensive empirical evaluation, encompassing 12 benchmarks across diverse distribution shifts, to showcase MANO’s superiority over 11 baseline methods. The results consistently show that MANO outperforms state-of-the-art baselines, even under challenging natural shifts.

Weaknesses

1. It does not address how to translate the proposed MANO into a practical estimated accuracy, which is crucial for real-world applications. Additionally, it should report the performance of the methods using the absolute estimation error metric, defined as the absolute difference between the estimated accuracy and the actual accuracy, for unsupervised accuracy estimation. 2. The justification for selecting $\eta=5$ is not robust. An ablation study on the impact of varying $\eta$ should be conducted to strengthen the argument. 3. It lacks a discussion of some key related works on unsupervised accuracy estimation, such as [1][2]. [1] Chen, Jiefeng, et al. "Detecting errors and estimating accuracy on unlabeled data with self-training ensembles." Advances in Neural Information Processing Systems 34 (2021): 14980-14992. [2] Chuang, Ching-Yao, Antonio Torralba, and Stefanie Jegelka. "Estimating Generalization under Distribution Shifts via Domain-Invariant Representations." International Conference on Machine Learning. PMLR, 2020.

Questions

1. Could the authors also report the performance of the approaches under the absolute estimation error metric? 2. Could the authors perform an ablation study on the effect of the hyper-parameter $\eta$? 3. Could the authors add the missing related works on unsupervised accuracy estimation?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have adequately addressed the limitations and potential negative societal impact of their work.

Summary

This paper proposes the OOD accuracy estimation method, named MANO, by leveraging the positive correlations between features to decision boundary distance and generalization performance. Along with Softtrunc for preventing error accumulation in overconfidence scenarios, the proposed method outperforms existing estimation baselines over various benchmark datasets.

Strengths

- This paper provides the theoretical insights that motivate the simple but effective OOD accuracy estimation method without supervision. - Analysis on extensive set of distribution shift benchmarks as well as ablation studies demonstrate that the proposed method shows promising results in various distribution shifts.

Weaknesses

1. The method requires hyperparameters, such as $p$ in aggregation step and determining model’s calibration for softrunc, which requires the OOD labels for optimal hyperparameter search. 2. As mentioned in Section D.2, the selection of criterion ($\Phi (D_{test})$) higher than $\eta$ could still include the high errors and high confidence (overconfindence), i.e., first scenario has high lower bound of $\Phi (D_{test})$ as in third scenario. Therefore, more thorough analysis or ablation studies on how sensitive MANO is under such potential failures in correctly determining calibration is required. In addition, miscalibration includes the underconfidence, where models have underconfident predictions compared to their actual accuracy. 2. Important estimation baseline is omitted, Agreement-on-the-Line, Baek et al. (2022). Baek et al., Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift, NeurIPS 2022

Questions

- Averaged confidence can be considered as particular case of MANO with softmax, i.e., using p as infinity. Still, in Table 4, even using Softtruc, AC is not competitive against MANO. So, what makes MANO, considering featuer-to-boundary for all classes, superior against AC, which corresponds to only considering the most confident (probable) class?

Rating

6

Confidence

4

Soundness

3

Presentation

4

Contribution

4

Limitations

See weaknesses and questions.

Summary

The paper presents MANO, a method for unsupervised accuracy estimation under distribution shifts. The method addresses the challenge of estimating model performance on out-of-distribution (OOD) samples without access to ground-truth labels. Firstly, the authors investigate the correlation between logits and test accuracy, and propose a novel approach that involves normalizing logits and using the Lp norm of the normalized logits matrix as an estimation score, motivated by the low-density separation assumption. Then, the authors find that the commonly used softmax normalization method tends to harm the performance estimation when facing overconfidence issue on some particular dataset. The authors demonstrate that MANO achieves state-of-the-art performance across various benchmarks and architectures.

Strengths

1. This paper introduces a novel method of using the norm of normalized logits for accuracy estimation, providing a fresh perspective on handling accuracy estimation problem under distribution shifts in unsupervised settings. This innovative approach builds on the low-density separation assumption, which is theoretically sound and practically relevant. 2. The authors consider the common overconfidence issue faced by deep models and propose a new normalization method based softmax operator. The illustrated experiments show that the new SoftTrun method shows improvement in some cases.

Weaknesses

1. Despite the authors designing a method based on the low-density assumption from theoretical perspective, this approach appears rather trivial and is quite similar to entropy-based methods like the variant of ATC. Moreover, it measures the overall output smoothness of the test set, making the connection with uncertainty somewhat trivial. 2. The proposed SoftTrun method, which is an improvement over softmax, shows performance enhancements in some cases (as shown in Figure 2(b)), but can also degrade performance in others. For new tasks, SoftTrun may not always outperform the traditional softmax method. Additionally, the experimental validation for this aspect is insufficient, as Tables 1 and 2 lack comparative results with the softmax-based MANO baseline method.

Questions

Please refer to the weaknesses.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

N/A.

Reviewer-Author discussion period

Dear Reviewers, The deadline for the reviewer-author discussion period is approaching. If you haven't done so already, please review the rebuttal and provide your response at your earliest convenience. Best wishes, AC

Final words to reviewers

As the discussion period is ending, we thank the reviewers for taking the time to review our paper and discuss it with us. We believe their valuable suggestions strongly benefitted our paper. We are glad that our clarifications and the additional experiments requested addressed the reviewers' concerns and are grateful to them for recognizing the core strengths of our work: - Novel and innovative - Theoretically well-motivated - SOTA efficient approach - Extensive comparisons We thank them again for their insightful feedback. Sincerely, The Authors