GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradients

Summary

The paper proposes to project the per-example parameter-gradients of the prediction head onto the orthogonal subspace of the learned features and uses the projection norm as the OODness score. The key motivation for doing so is that the parameter gradients lie in the subspace of the corresponding input features. Key highlights of this method are that it is a post-hoc detection method and is computationally efficient requiring few in-distribution examples for computing the orthogonal subspace of the features. It achieves good detection scores.

Strengths

The main strength of the paper is in the idea building upon the observation that stochastic-gradient descent updates lie in the subspace of input features for detecting OOD examples. The paper is very well written and the reports various ablation studies to clearly illustrate the effects of the hyperparameter design-choices.

Weaknesses

While the empirical results show good performance on the commonly considered benchmarks, I believe that it is essential to incorporate results related to Semantic Outliers [1] and Near-distribution Outliers, such as the comparison between CIFAR10 and CIFAR100 [2, 3]. Section 4.3 in [4] can serve as an additional reference for conducting a similar evaluation. It is worth noting that it is okay if the results are not very strong; rather, the key objective is to present a comprehensive assessment of the estimator and update the limitations section accordingly. Furthermore, it would be beneficial to include a clear rationale behind the selection of baseline algorithms -- considering a comparison with [3] would be appropriate if the primary empirical contribution focuses on far-OOD detection. I also thought that a qualitative analysis of the estimator could also be helpful in illustrating the cases where GradOrth works best and its failure cases as compared to other baseline estimators. [1] Ahmed, F., & Courville, A. (2020). Detecting Semantic Anomalies. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 3154-3162. [2] Fort, S., Ren, J., & Lakshminarayanan, B. (2021). Exploring the limits of out-of-distribution detection. Advances in Neural Information Processing Systems, 34, 7068-7081. [3] Sastry, C. S., & Oore, S. (2020, November). Detecting out-of-distribution examples with gram matrices. In International Conference on Machine Learning (pp. 8491-8501). PMLR. [4] Hsu, Yen-Chang, Yilin Shen, Hongxia Jin, and Zsolt Kira. "Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10951-10960. 2020.

Questions

1. I believe that this work has some connections with [1, 2] and can be used to complement the abilities of GradOrth in interesting ways. 2. In Appendix B, you show that the gradients obtained from all layers perform a bit worse than the final layer alone: do you think that using a different scaling factor for each layer can help in building a better estimator than just the last layer? 3. The algorithm requires computing per-example gradients of the loss wrt parameters: how did you do this for processing a batch of examples while staying as computationally efficient as possible? A short discussion about this would be useful to include in the paper. 4. Apart from the obvious typos in the equation on line 124 of subsection 2.2, could you please elaborate why you compare the frobenius norms of matrices ${\bf R}$ and ${\bf R}_k$ instead of the obvious choice of using the difference $|| {\bf R}_k - {\bf R}||_F$ for identifying the appropriate $k$? Is this related to the inequality between frobenius norms and squared sum of eigenvalues -- if so, it would be good to explain this. 5. In section (3), it would be good to include the shapes of matrices wherever relevant. 6. If you have any preliminary results/thoughts about the weaknesses section, please include it in the replies. (The new experimental results highlight the effectiveness of the method and I raise my score.) [1] Koh, Pang Wei, and Percy Liang. "Understanding black-box predictions via influence functions." ICML 2017. [2] Madras, David, James Atwood, and Alex D'Amour. "Detecting underspecification with local ensembles." ICLR 2020.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Yes, the limitations are adequately addressed.

Summary

The work proposes GradOrth, a simple post-hoc OOD detection method that seeks to improve on existing gradient-based OOD detection approaches by leveraging a selected lower-rank subspace of the gradient most relevant for OOD detection. The resulting GradOrth method shows superior OOD detection performance on a wide range of OOD detection tasks.

Strengths

1. The paper is clearly written and investigates a generally underexplored aspect of OOD detection. 2. The resulting GradOrth post-hoc method is easy to implement and shows strong empirical performance on a wide range of OOD detection tasks. 3. The work presents a series of relevant ablation studies ranging from the choice of norm to the number of ID samples used to compute the subspace.

Weaknesses

1. Some comparable and competitive methods are missing. For example, the reviewer would like to see empirical results comparing DICE [1] with GradNorm [2] and KNN [3]. 2. Additionally, the reviewer would like to see empirical result deviations, as randomness with respect to the chosen ID samples, used to compute the subspace, may be relevant to the empirical performance of GradOrth. [1] Yiyou Sun and Yixuan Li. Dice: Leveraging sparsification for out-of-distribution detection. In Proceedings of European Conference on Computer Vision, 2022. [2] Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. In Advances in Neural Information Processing Systems, 2021. [3] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International Conference on Machine Learning, 2022.

Questions

1. Is there a reason why DICE's sparsification can not be implemented alongside a method like GradNorm? 2. Previous attempts at Gradient-based approaches have shown L1 norm to be superior, do the authors have some intuition on the reasoning why L2 norm is stronger for GradOrth?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

The authors did not directly address societal impacts however there should be no negative impacts from this work.

Summary

The paper proposes an OOD detection method by exploiting the orthogonal projections of gradients to a subspace. Once a model is trained on an in-distribution (ID) dataset, the k most significant features in the last layer of a network are found using SVD using a small subset of n ID samples. Then, the gradient of the loss given an input data w.r.t. the parameters of the last layer are projected to this significant subspace. Finally, the norm of this projection is used as OOD score which is expected to be large of OOD samples and low for the ID ones. The experiments are performed on 3 ID datasets (ImageNet, CIFAR10 and CIFAR100) with different architectures and OOD datasets. The results demonstrate that the proposed method achieves the highest average scores across all OOD datasets for each ID dataset.

Strengths

- The idea of using projection of gradients for OOD detection is interesting and, to the best of my knowledge, is novel. - The paper presents the idea quite clearly and supports many of the claims (there is one exception I will mention in the weaknesses section) with sufficient experimental results. - The paper acknowledges the fact the performance of many OOD methods varies a lot in different datasets and mentions that the performance of the method holds for the datasets used in the paper.

Weaknesses

- The idea presented in the paper is interesting and the paper presents sufficient experiments on the common OOD benchmarks in the literature. However, the results of almost all methods are already very close to each other in this benchmark and there is no clear winner, as also mentioned in the paper. Some baseline methods may perform slightly better than those reported in the paper with a more careful hyperparameter tuning. For example, the original Mahalanobis paper reports AUROC of 99.2 on LSUN dataset when CIFAR10 is in-distribution while the paper reports 97.09 in Table 2. Therefore, OOD detection methods should be evaluated on more realistic and challenging scenarios to better assess their performance, such as in near-OOD detection settings as in [1] where CIFAR10 is in-distribution data and CIFAR100 is OOD and vice versa. The paper lacks an evaluation of the methods in such a more realistic and challenging setting where most OOD detection methods fail. Even if the results in this setting are not good, this should be presented as a limitation of the work. [1] Lee et al. A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks - The paper claims that the method has a lower computational complexity than existing methods such as Mahalanobis. However, this claim is not supported by quantitative results. As the paper claims, extracting features at intermediate layers in Mahalanobis may not bring significant computational complexity.

Questions

- Given the similar performance of all OOD methods on this benchmark, why should one use the proposed method instead of the others? - How does the method perform in a near-OOD setting? - Is the computational time of the proposed method significantly lower than the other methods, as claimed in the paper? Please provide quantitative results.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The paper discusses some limitations of the evaluation of OOD methods in general rather than the limitations of this method. I don't see any potential negative societal impact of this work!

Summary

This paper focuses on OOD detection based on gradient information. It seems based on the framework of GradNorm (the paper is confusing on this) to use the gradient response as the score for detecting OOD data. Specifically, it uses SVD based subspace modeling and projection of the testing sample’s gradient as the OOD detection score. As shown in the reported results, the proposed method performs well and achieves better performance on many measurements compared to the previous methods.

Strengths

- Generally, it is well-motivated to use subspace analysis to enhance the gradient-based OOD detection approach. - Experiments are conducted comprehensively on different benchmark datasets and with different backbone models.

Weaknesses

- The formulation of the approach is not clearly introduced or missed, which is the basis of the work. This makes the paper very confusing and influences the understanding of the model’s behavior. The details are in the following: - How is gradient calculated on the testing samples when only input images are available in inference time? Does the paper use the model formulation similar to GradNorm (or ExGrad), which uses the gradient from the loss by letting the model predict uniform probability in testing? Or the proposed model uses some different strategies maybe related to the formulations in sec 2.1 (seems not make sense)? No matter what, the authors may provide a rigorous formulation and discuss the technical motivations scientifically. - This influences understanding and evaluating the specific motivation and technical soundness of the subspace modeling. Is the modeling focused on the subspace of representation, gradients, or the relationship of them? Since the subspace $S^L$ obtained in Algo 1 is for the representation, understanding how the gradient is calculated in Algo 2 is important. - The evaluations are with some problems, with details as the following points: - There are several operations involving randomness. I cannot find whether the paper evaluate how the randomness may influence the performance and may report the mean and std of the results while given different random seeds. - Specifically, the ID representations and SVD-based subspace models are from selected samples. The results should be sensitive to the quality of the selected ID samples, especially when the number is small. I can see experiments studying how the number of selected samples may influence the results. How the randomness may influence the selection and then the results? - The authors may consider analyzing and evaluating the distribution of the scores of ID and OOD samples, as in previous works. ---after rebuttal--- The proposed method does make sense and performs slightly better than the main competitors, without supersize. It is a combination of several existing methods, such as GradNorm and the projection-based methods of CL. I didn't mention this issue since the representation in the original paper is very unclear and confusing, where the issues are hidden in the unclearly stated formulation. I can and only can increase the score to Borderline accept.

Questions

The main questions are left along with the weakness points. How is the computational efficiency of Algo 1 and how does the number of selected ID samples influence the time?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

1 poor

Contribution

2 fair

Limitations

There is a paragraph discussing limitations in the paper, mainly focusing on the general limitation of OOD detection methods. The authors may enhance it by focusing on the proposed techniques, such as how the selected ID sample may influence the performance, and whether selecting more ID samples may influence the computation time.

Summary

A method is proposed to perform out-of-distribution detection on a pre-trained classifier. The approach used by the authors is to utilize a low-rank approximation of the parameter space for in-distribution data. Gradients for in-distribution samples are more likely to be projected directly onto the parameter subspace, whereas OOD gradients are more likely to be orthogonal and have small projection values. The proposed method is simple to implement and results in impressive performance compared to other methods for out-of-distribution detection

Strengths

1) The proposed method has low computational complexity and be easily applied to any pre-trained model. 2) Impressive performance on a comprehensive benchmark. 3) Good ablations performed to investigate robustness of the method and effects of different method components.

Weaknesses

1) The specified threshold $\epsilon_{th}$ is not discussed in very much detail. It is unclear if the model performance depends on heavily tuning this value. 2) Experiments are performed on data with significant differences between ID an OOD data. It would be valuable to assess results with CIFAR10 and CIFAR100 as an ID / OOD pairing as in "Exploring the Limits of Out-of-Distribution Detection". 3) Writing and notation could be improved in section 2 and 3 (see questions for more comments)

Questions

1) How is this threshold $\epsilon_{th}$ selected, it is constant across datasets? How robust is the model as this parameter is varied? It would beneficial to include the value of $k$ in the results table descriptions. 2) What is the effect of batch statistics on the performance of your method? For example, ResNet uses batch normalization, meaning that the ratio of ID and OOD samples during test time will have an effect of the model predictions. How are batch statistics controlled in your experimentation? Notation: 2) In section 2.3, is there a reason why $O(\mathbf{x})$ was selected? It might be worth changing this to a different letter to prevent confusion with big O notation. 3) In section 2.4 the typical notation is to represent vectors with lowercase boldface. Is there a reason for using alternative notation? 4) In section 3 the acronym ID and OOD are introduced multiple times in this section.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Limitations are not discussed in detail. It is recommended to include a section in the Appendix to discuss this.

Response to Rebuttal.

Thank you for your extra experiments and answers to my questions. I think that it will be good to add some of this discussion into the paper.