Handling out-of-distribution (OOD) samples has become a major stake in the real-world deployment of machine learning systems. This work explores the use of self-supervised contrastive learning to the simultaneous detection of two types of OOD samples: unseen classes and adversarial perturbations. First, we pair self-supervised contrastive learning with the maximum mean discrepancy (MMD) two-sample test. This approach enables us to robustly test whether two independent sets of samples originate from the same distribution, and we demonstrate its effectiveness by discriminating between CIFAR-10 and CIFAR-10.1 with higher confidence than previous work. Motivated by this success, we introduce CADet (Contrastive Anomaly Detection), a novel method for OOD detection of single samples. CADet draws inspiration from MMD, but leverages the similarity between contrastive transformations of a same sample. CADet outperforms existing adversarial detection methods in identifying adversarially perturbed samples on ImageNet and achieves comparable performance to unseen label detection methods on two challenging benchmarks: ImageNet-O and iNaturalist. Significantly, CADet is fully self-supervised and requires neither labels for in-distribution samples nor access to OOD examples.
Paper
Similar papers
Peer review
Summary
This paper consists of two mostly unrelated parts. In the first part, the authors aim to detect whether two sets of samples are drawn from the same distribution. To do this, they apply the unbiased estimate of the MMD, in the manner suggested by Sutherland et al., to produce a conformal p-value. Then they propose a small variation on Sutherland's approach, which they call MMD-CC, that requires twice as many samples but supposedly reduces some variance (although no further explanation or analysis is given). They apply these techniques to detecting distribution shift between CIFAR-10 and CIFAR-10.1 using a feature-generating network trained on an external dataset (ImageNet), similar to "outlier exposure" methods. In the second part of the paper, they aim to detect whether a single sample is drawn from same distribution as a set of inliers or not. For this, they propose the CADet method, which has no discernable relationship to MMD (3) or the unbiased MMD estimate in (4). Rather, CADet is closer to the CSI approach from Tack et al. and the ICLR'21 approach of Sohn et al., in that it relies on contrastive learning and distributional shifts. Because key parts of CADet are never explained by the authors, it's difficult to summarize further. The authors test CADet on experiments where ImageNet is the inliers and other datsets are the outliers, but they never compare to the state-of-the-art CSI approach even though they write "the closest work in the literature is CSI" on line 86.
Strengths
The combination of Sutherland's MMD with SimCLRv2 gives good empirical performance in detecting distributional shift between CIFAR-10 and CIFAR-10.1.
Weaknesses
1. Claims about the existing literature are overblown and key citations are missing. a. On line 46, the authors claim that "only a few works detect OOD samples from single inputs ... without requiring access to OOD samples to train or tune the method." But there are dozens, if not hundreds, of papers proposing such methods. b. Although the authors acknowledge that the CSI paper by Tack et al. is very close in spirit to CADet, they do not cite an even closer method in the ICLR'21 paper by Sohn et al. c. Although the authors frequently compute conformal p-values, they don't cite any relevant literature. d. The authors do not cite previous work that uses MMD for OOD, such as Dong et al.'s "Neural Mean Discrepancy for Efficient Out-of-Distribution Detection". 2. Throughout the paper, the p-values are invalid. In the best case, the conformal p-values computed by the authors hold only only marginally, i.e., in expectation over the validation data. They do NOT hold for a fixed validation set. For a detailed explanation, see Bates et al. "Testing for Outliers with Conformal p-values" or Magesh et al. "Multiple testing framework for out-of-distribution detection" or Angelopoulos et al. "Conformal Prediction: A Gentle Introduction". 3. In some cases, the p-values are invalid for more fundamental reasons. In MMD-CC Algorithm 1, for example, the conformal p-values are computed as if n_perm was the size of an i.i.d. validation dataset. But n_perm permutations are not independent and so the p-values computed in this way are invalid. For example, there is nothing preventing the user from specifing a value of n_perm that is higher than the total number of possible permutations. In fact, the MMD-CC method is entirely heuristic and comes with no analysis, which is a serious problem. 4. The MMD-CC method is underwhelming at a practical level as well. First, it requires twice as many samples as Sutherland's MMD. Despite that limitation, it performs worse in Table 1. 5. For the CIFAR-10-vs-CIFAR-10.1 experiment, the proposed feature generators were trained using ImageNet (see line 160). This is problematic for several reasons. First, using an external datset for OOD detection is a form of outlier exposure that violates the main claim of the paper stated on line 45. Second, the other methods under test do NOT use an external datset, making the comparison unfair. Third, the use of an external dataset goes against the instructions on line 94, which explicitly state that SimCLRv2 is to be trained on "in-distribution samples." Fourth, alhough the authors wrote that "a fair comparison is difficult," that's not true; they could have easily trained their feature generator on the CIFAR-10 inlier dataset and it would have avoided all these problems. 6. The authors claim a weakness of CSI is that it can't detect adversarial perturbations, but in doing so the authors are conflating two entirely different versions of the OOD problem. The first version is where one is given a trained supervised classification network and asked to detect samples different from those used to train it. In this problem, we start with a network and so adversarial perturbations are well defined. The second version (solved by CSI) is where one is given an inlier distribution and are asked to detect samples different from it. Here, there is no network, and thus no adversarial perturbations, and so it is unfair/illogical to claim that CSI is "not well suited for adversarial detection." CSI solves an entirely different problem. 7. The section "detection of distribution shifts from a small number of samples" hides the fact that a huge number of samples are required to train the feature generating network. In fact, by training on an external dataset, the authors are implementing a form of "outlier exposure" that goes against the main requirement stated on line 45. 8. Critical information about the CADet method (the main contribution according to the title/abstract) is missing, making it impossible to fully evaluate the paper. In particular, the similarity function "s" in (5) is never specified. On line 195 it says "self-supervised contrastive learning trains a similarity function," but that is incorrect: SSCL trains a feature generating network that maps an input "x" to a set of features "z," as specified on line 99. SSCL does NOT train a similarity function that maps two inputs "x1" and "x2" to a scalar. 9. The choice to linearly combine the m^in and m^out scores in (6) is heuristic, and probably highly suboptimal. The framework of multiple hypothesis testing gives principled ways to combine scores. For more on multiple hypothesis testing, see Candes' STATS 300C lecture notes. 10. The CSI method by Tack et al. is "the closest work in the literature" according to line 86. Yet the authors curiously omit it from all the numerical experiments, choosing much easier competitors.
Questions
1. Regarding the perturbations in Table 2, they are never clearly defined. Are these perturbations of the Supervised ImageNet-trained classifier used in the bottom two rows? 2. There seems to be a mistake in the contrastive loss equation (1), since the "j" in the numerator is undefined. Should the numerator be "u_{i,i}"?" 3. There seems to be a mistake in the CADet definition (5). The denominator of m^out contains the number of terms in the numerator sum, but the denominator of m^in does not. Should the latter be "ntr(ntr-1)"?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
yes
Summary
This paper studies the OOD detection of new classes and adversarial attacks with learned similarities via self-supervised contrastive learning in conjunction with MMD two-sample test. To enable MMD applicable for OOD detection on single samples, they improve the idea by using augmentations to create a set. Experimental results on new classes and adversarial examples detections show that their method can perform favorably against previous arts.
Strengths
1. The paper is well-written and well-organized. 2. The paper does a good job summarizing preliminary ideas and providing detailed settings reproducing the overall algorithm. 3. The paper provides helpful discussions regarding the efficiency of the method.
Weaknesses
1. According to Table 3. it seems that the proposed method does not show superior performance against previous arts when using supervised training. Discussions regarding this matter could be included to better justify the method. 2. I'm not an expert on OOD detection. I'm wondering why there are no error bars in the reported results, given that different hyper-parameters could easily improve (or decrease) the performance, and the differences between the compared arts are not that significant reported in Table 3. 3. Some hyper-parameters are empirically selected without proper justifications, especially those regarding augmentation.
Questions
See weakness
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
Limitations have been discussed in the manuscript.
Summary
The authors propose to handle the OOD detection problem, exploring self-supervised contrastive learning to detect samples from previously unseen classes and adversarially perturbed samples. They use self-supervised contrastive learning with the maximum mean discrepancy (MMD) to test if two sets of samples originate from the same distribution. They propose CADet which takes advantage of the similarity of the contrastive representation/transformation of the same sample. The proposed approach is thoroughly tested in two scenarios: the scenario with previously unseen classes and the scenario with adversarially perturbed samples.
Strengths
+ The problem of OOD detection is very important in a lot of aspects of machine learning, and it's a worthwhile problem to study and research. + The proposed approach, CADet, achieves good performance on the benchmarks. + The combination of MMD and self-supervised contrastive learning seems to be effective.
Weaknesses
- The work is motivated by the claim that OOD samples will be encountered in the wild when the machine learning models are deployed in the wild, but this work is not tested in the wild, in a smart computing or cyber-physical system setting. This is a minor complaint as it is tested on (solely) datasets, but it is advised that the authors revise the paper to not include such claims. - The work fails to be evaluated on "fake" OOD samples, or samples that might originate from a different distribution than the training samples but belong in the same classes. For example, the Webcam domain in the Office-31 dataset and the Amazon domain in the same dataset, have the same classes, but distribution-wise, the samples are different. Will CADet consider Webcam's samples of class A to be OOD from Amazon's samples of class A? - The Related Works section is too high-level: it's evident that the authors are aware of the recent advance in the field of OOD detection, but a more detailed explanation (one sentence or two) for (most of) the mentioned works would be desirable.
Questions
+ What is the intuition behind the fact that self-supervised contrastive learning works well with MMD?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
2 fair
Contribution
3 good
Limitations
+ The paper proposes an effective approach (self-supervised CL + MMD), but there are other mainstream alternatives to MMD such as the KL divergence and Mahalanobis distance. Why do you pick MMD in particular, instead of KL-divergence and Mahalanobis distance (both of which can be adapted to perform on both instance-level and distribution-level)? Is there any comparison on why KL-divergence based and Mahalanobis distance based approaches aren't as effective?
Summary
The paper proposes Maximum Mean Discrepancy with Clean Calibration (MMD-CC), as an improvement of MMD when the number of samples is small. Moreover, it introduces CADet, a novel anomaly detector for both OOD and adversarial detection inspired by MMD.
Strengths
In line with recent works, unsupervised anomaly detection in deep learning is a relevant research objective. The paper is well-written and proposes a novel approach for anomaly detection, improving existing works on Maximum Mean Discrepancy. Moreover, I appreciated the idea of improving MMD to overcome its limitation of requiring sets of test samples.
Weaknesses
- Experimental setting. I did not understand why for benchmarking adversarial detection you kept ODIN [1] and discarded the Mahalanobis detector. To the best of my knowledge, ODIN is mainly designed for OOD detection, furthermore in [2] the Mahalanobis detector achieves better performance than both LID [3] and ODIN for adversarial detection. Moreover, no adaptive attack was tested [4]. - Strength of the results. In Table 3, CADet has a high variance of the results depending on the training strategy (iNaturalist vs ImageNet-O) and is not the overall best performing. Moreover, only the AUROC was reported, while other commonly employed metrics are missing (FPR, AUPR). Regarding MMD-CC, it should be better clarified its significance given that it is significantly better only on Table 2, with PGD and small n_samples. - Missing related works. Recently, other unsupervised anomaly detection (for both OOD and adversarial) algorithms have been proposed, such as [5]. A comparison with such method should be performed. - Reproducibility. No code was released. Some hyperparameters are missing (e.g., number of neighbours for LID [3], perturbation size for Mahalanobis [2] and ODIN [1]). Given how challenging it is to evaluate and compare OOD detectors, this is an important weakness. - Architecture scale. Given the existence of other unsupervised methods for anomaly detection ([5]) and the lack of convincing performance improvements over related methods, the computational limitations of CADet are a significant weakness. [1] Liang, S., Li, Y., & Srikant, R. (2018). Enhancing the reliability of out-of-distribution image detection in neural networks. ICLR 2018. [2] Lee, K., Lee, K., Lee, H., & Shin, J. (2018). A simple unified framework for detecting out-of-distribution samples and adversarial attacks. NeurIPS 2018. [3] Ma, X., Li, B., Wang, Y., Erfani, S. M., Wijewickrema, S. N. R., Schoenebeck, G., Song, D., Houle, M. E., & Bailey, J. (2018). Characterizing adversarial subspaces using local intrinsic dimensionality. ICLR 2018. [4] Tramer, Florian, et al. "On adaptive attacks to adversarial example defenses." Advances in neural information processing systems 33 (2020): 1633-1645. [5] Raghuram, J., Chandrasekaran, V., Jha, S., & Banerjee, S. (2021). A General Framework For Detecting Anomalous Inputs to DNN Classifiers. Proceedings of the 38th International Conference on Machine Learning, 8764–8775.
Questions
### Suggestions: Given the amount of OOD and adversarial detectors now available, I would expect a strong validation of a method to prove its effectiveness. I think that the most important concern that should be addressed is how CADet, given its resource limitations, can be a significant improvement over existing unsupervised OOD/adversarial detectors such as [1]. ### Typos: - Line 140, "Additionnally". [1] Raghuram, J., Chandrasekaran, V., Jha, S., & Banerjee, S. (2021). A General Framework For Detecting Anomalous Inputs to DNN Classifiers. Proceedings of the 38th International Conference on Machine Learning, 8764–8775.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The authors addressed the main limitations of their method.
Summary
The paper presents a novel method for Out-of-Distribution (OOD) detection called CADet (Contrastive Anomaly Detection). The authors leverage self-supervised contrastive learning and the maximum mean discrepancy two-sample test(MMD) to assess whether two sets of samples have been drawn from the same distribution. The method is designed to detect OOD samples from single inputs and performs well on both label-based and adversarial detection benchmarks, without requiring access to any OOD samples nor previous classes to train or tune the method.
Strengths
1. The authors propose a novel method for OOD detection that does not require access to neither in distribution samples nor OOD samples to train or tune the method. 2. They use similarity functions learned by self-supervised contrastive learning with MMD to assess the distribution even with few shot images. 3. Paper shows the method outperforms current methods in adversarial detection tasks while performing well on label-based OOD detection.
Weaknesses
I do not see major weaknesses for this paper as of now. I will be updating this section if need be later.
Questions
1. I understand ResNet50 is a popular architectures that has been used for OOD and adversarial benchmarks. But there has been significant developments in those areas too, did you have a chance to see how would the method perform on other architectures beyond ResNet50?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
4 excellent
Contribution
3 good
Limitations
Authors have done a good job presenting the limitations of their paper.
After reading the rebuttal, I have some additional concerns/questions: 1. Your design choices on the Mahalanobis and LID detectors should be better clarified. You should explain that you kept only the last layer for the Mahalanobis distance (following the ViM method); indeed, as you correctly stated in the rebuttal, it makes sense to extend it in the adversarial scenario. Given this new experimental details, I wonder if also for LID you kept only the last layer, since otherwise it would have required to train a classifier. At least to me, these details were non-obvious during the review and should be clarified (at least in the Appendix). 2. The lack of adaptive (or an analysis with various, smaller, perturbations sizes) makes the evaluation on adversarial attacks limited in comparison to other detectors (like JTLA), given also the fact that it is one of the main selling points of CADet. I think this should be (briefly) stated in the conclusions/limitations. Regarding the other issues: 3. I think that the reproducibility concerns, excluding 1 above, are addressed. In particular, I thank the reviewers for detailing their status with regard to the code publication issue. 4. Since no clear difference among the metrics (AUROC, ...) was detected, I agree on favouring readability over completeness. 5. Regarding the JTLA method, I acknowledge that it would have required some adaptations (fitting aklpe on a subset of ImageNet), since it was tested only on smaller training sets. Regarding related works, I would also reference [1], that does not require labelled adversarial and scales to Imagenet. Of course I do not expect further testing on it given the restricted timeframe. 6. Regarding the performance of CADet on novel class detection, I would leave the results section and the overall presentation as it is (CADet as a -generalized- anomaly detector), given my concerns on the evaluation of adversarial attacks. 7. I would state better that MMD-CC shows significant improvements (at a computational cost) mostly on PGD, since in the other cases performs essentially on par with MMD (apart from iNaturalist, n=3). Minors: - citations in methods of table 4. - missing citations of adversarial detectors employed in the results in the related works section, such as LID ([40]). To summarise, after re-reading the work and the rebuttals, I think that the work contains valuable contributions. However, I have still have some doubts on the performance improvements over SOTA (in particular with regard to adversarial attacks), together with the computational limitations. I am willing to increase the rating after the response by the authors to my additional observations. [1] "The Odds are Odd: A Statistical Test for Detecting Adversarial Examples", Roth K. et al., ICML 2019.
We thank the reviewer for their careful consideration of our rebuttal, and more generally of our submission. 1. Indeed, we only compute nearest neighbours based on the last layer, and then directly use it as a detection score. The last layer was found in the original paper ([1], Figure 2) to be the most useful to directly perform detection. However, as the reviewer noted, this is not specified anywhere in our draft and we simply refer to the original paper which mainly investigates the version of their algorithm training a classifier on all layer scores. Similarly, while it has been fairly common in the field to use the Mahalanobis distance of the last layer as a detection metric, we cite the original paper without specifying how we perform computation. We will add in appendix a section focusing on a precise description of our implementation of recent methods, with full hyperparameters and such adaptations. 2. We will add a subsection in the limitation section focusing on discussing adaptive attacks. Our viewpoint is that defence methods, including detection-based, are always weak against adversarial attacks. [2] showed that many recent decent methods claiming robustness to adaptive attacks are actually weak against well-designed adaptive attacks. In general, recent work can only show the robustness of proposed methods against adaptive attacks of their design. It has become a consistent trend in the literature to subsequently observe that they are in fact not robust to properly design adaptive attacks. Given these elements, we find that even if we found our method to be robust against an adaptive attack of our design, it would be insufficient to claim robustness to such type of attacks. Realistically, it is safe to assume that CADet is not robust to adaptive attacks, like most recent works, and it is nearly impossible to prove that other detection methods possess such robustness anyway. Nonetheless, there is still practical motivation to seeking such detectors: while it is well known that adversarial attacks transfer relatively well across CNN architectures, adaptive attacks against detectors require very specific knowledge of the attacked detector. As such, attacking an unknown CNN-based classifier remains fairly easy, while trumping a detector requires knowledge of the model, adding protection. 5- We were not aware of this work, which indeed seems extremely relevant. We will investigate it in detail, cite it, and if possible produce comparisons before camera-ready deadline -- should our submission be accepted. Minors: we will fix these issues [1] Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality, *X Ma et al.* [2] On Adaptive Attacks to Adversarial Example Defenses, *F Tramer et al.*
1. Thank you for the additional details, that I think are necessary to have a full understanding of your experimental setting. Indeed, in my experience, I always considered the multi-layer implementation of both LID and Mahalanobis. 2. Given that your framework is not meant to be a defence against adversarial attacks, but rather a new perspective on unsupervised (generalized) OOD detection, I acknowledge that adaptive attacks can be skipped. Although, given the literature in this regard and the expected experimental settings, I would briefly discuss it as you plan to do. The final version of this paper will require an important revision with regard to reproducibility, comparison with SOTA, and clarity. But the authors addressed all of my concerns and demonstrated their willingness to improve their work. Consequently, I will raise my soundness and contribution scores from 2 to 3 and my rating from 3 to 5.
We would like to thank again the reviewer for their time reviewing our submission and considering our rebuttal. We will make sure to carefully incorporate all suggested improvements, and promise the code will be fully released by publication date, should our submission be accepted.
Thank you for your rebuttal, which clarified several points in the paper that I missed when reviewing. As suggestions to improve the final version (if accepted): 1c. The paper would benefit from describing the p-value after L225 as a conformal p-value and citing relevant literature. 3\. The paper would benefit from an explanation/proof of why the proposed MMD-CC test outputs valid p-values. 8\. I still think it is confusing to say that SSCL trains a similarity function when the similarity function (a cosine) is fixed. Some rewording would help. 10\. Regarding the lack of comparison to CSI, I still believe that its absence significantly weakens your paper. The OpenOOD paper says only that it took longer than 48 hours to evaluate. In that work, this excuse is acceptable because they compare dozens of methods on 9 different tasks. But your paper has far fewer methods and fewer tasks, and you claim that CSI is the closest approach in the literature. So I don't think it's acceptable to omit CSI. Furthermore, your paper gives no justification for omitting CSI (only your rebuttal does). In any case, I have modified my review as follows: soundness = 3, presentation = 3, contribution = 3, overall = 5
We thank again the reviewer for their time reviewing our submission and consideration of our rebuttal. We will incorporate the suggested improvements. Regarding MMD-CC, we will add an appendix section with a formal proof of validity of computed p-values. Regarding the similarity function, we do not mean the cosine alone, but the composition of the cosine with the representations, that is, $f(x_1,x_2)=cosine(h(x_1),h(x_2))$ where $h(x)$ computes the representations for image $x$. Thus this function is not fixed, and admits the same parametrisation as $h$. We will add a full sentence specifying precisely our terminology. For CSI, we agree with the reviewer that it appears curious to omit evaluation when we describe their method as the closest in the literature. We will add a section in the appendix to give additional details on previous works (that would not fit reasonably in the related work section), such as our specific implementations and hyperparameter tuning for comparisons (as suggested by reviewer X9Xp), but also to justify our choice not to include CSI. In term of computations, scaling CSI to ImageNet with ResNet50 is not just longer than 48h. The training loss of CSI requires, on top of the usual contrastive transformation, to also augment each sample with 4 rotations (0, 90, 180, 270 degrees). The number of forward passes and backprops to perform for each step is effectively multiplied by 4. Training a ResNet50 on ImageNet with SimCLR took us over a week of training on 8 A100. Thus multiplying step complexity by more than 4 is not reasonably feasible with our computational means. This is without taking into account the addition of an extra linear layer + softmax objective, and the memory overhead which requires reducing the number of samples per batch (which, according to both original SimCLR papers, lead to performance drops). At inference time, CSI requires to compute nearest neighbor (in term of L2 distance between representations) wrt the whole training set. Of course, this step is very expensive with ImageNet. One can consider only a subset of the training set, but with a 1000 different classes, one can expect a large performance drop if the subset is reasonably sized. Finally, as stated L91-L93, CSI multiply their scores by the norm of representations. Unsurprisingly, performing detection based on representation norms alone leads to worse-than-random performances on successful adversarial attacks, which tend to have higher representation norms than inliers. Thus CSI is not reasonable for adversarial detection (though it would be very interesting to compare to CSI for novel class detection, if our hardware permitted it).
Decision
Accept (poster)