Summary
A novel method for identifying OOD samples is presented using Hopfield boosting. A classifier is trained using a loss function which sharpens the decision boundary between inlier and outlier data samples, where outlier data samples are purposefully drawn from an auxiliary dataset. By sampling examples close to the decision boundary and determining the modern hopfield energy of the sample, the decision boundary is adjusted to exclude OOD samples from the auxilliary dataset. The method is demonstrated twice using a low resolution and high resolution example. In both examples, the Hopfield boosting method outperforms state of the art on average. An extensive appendix includes additional background, theoretical results, and experimental results.
Strengths
The method is original, theoretically sound, and useful. The results are clearly presented, to the point where a reader may be able to implement the algorithm without referencing the provided code. The research question is impactful, and the result presented in this paper is a theoretical and experimental advancement in the field.
Weaknesses
The primary weakness is that the paper does not conclusively show that the method does not lead to overfitting. The models are trained with a classification loss, but the authors state "the toy example does not consider the inlier classification task that would induce secondary processes, which would obscure the explanations." However, failure to report these results in the context of the decision boundary refinements shown in Figure 2, 3b and 6 opens the work to questions of model overfitting.
This concern is only partially lifted by near perfect results on the test datasets. Because the false positive rate (FPR) is reported with respect to an OOD test dataset, there is a question of whether the model is losing in-distribution detections. Robustness to OOD samples of the original dataset, robustness to adversarial instances, and catastrophic forgetting are all implicated in the sharpening of decision boundaries; this work avoids acknowledging these cases and prevents inference about the implications by omitting the classifier performance for the trained models. This can be resolved by reporting the model F1 scores in the appendix for the ID classification task. If the models were trained using the Hopfield Boosting alone, then this would not be necessary. But the presence of the classifier loss means the secondary effects have implications for the semantic meaning of the decision boundary.
Questions
1. How do these results compliment/contrast with results suggesting that a smooth decision boundary is more robust to adversarial samples? See for example [Anderson (2022)](https://proceedings.mlr.press/v168/anderson22a)
2. Most computer vision datasets are expected to have some features in common, which motivates transfer learning techniques. What justification was used to determine that the test datasets would be strictly OOD w.r.t. the training datasets CIFAR or ImageNet?
3. Is there any analysis which shows that the decision boundaries sharpened during Hopfield Boosting are semantically meaningful, and not based on noise of the dataset?
4. The ResNet-50 model is fine-tuned rather than trained from scratch; what dataset was used to train the pre-initialized weights? Why was the model not trained from scratch? Because the feature space was already formed (4 epochs is not enough to significantly shift the model), does this imply that the model classification layer is likely identifying noise?
Limitations
Please see the questions above. The remaining limitations are thoroughly addressed in the work.