Enhanced Isotropy Maximization Loss: Seamless and High-Performance Out-of-Distribution Detection Simply Replacing the SoftMax Loss

Current out-of-distribution detection approaches usually present special\nrequirements (e.g., collecting outlier data and hyperparameter validation) and\nproduce side effects (e.g., classification accuracy drop and slow/inefficient\ninferences). Recently, entropic out-of-distribution detection has been proposed\nas a seamless approach (i.e., a solution that avoids all previously mentioned\ndrawbacks). The entropic out-of-distribution detection solution uses the IsoMax\nloss for training and the entropic score for out-of-distribution detection. The\nIsoMax loss works as a drop-in replacement of the SoftMax loss (i.e., the\ncombination of the output linear layer, the SoftMax activation, and the\ncross-entropy loss) because swapping the SoftMax loss with the IsoMax loss\nrequires no changes in the model's architecture or training\nprocedures/hyperparameters. In this paper, we perform what we call an\nisometrization of the distances used in the IsoMax loss. Additionally, we\npropose replacing the entropic score with the minimum distance score.\nExperiments showed that these modifications significantly increase\nout-of-distribution detection performance while keeping the solution seamless.\nBesides being competitive with or outperforming all major current approaches,\nthe proposed solution avoids all their current limitations, in addition to\nbeing much easier to use because only a simple loss replacement for training\nthe neural network is required. The code to replace the SoftMax loss with the\nIsoMax+ loss and reproduce the results is available at\nhttps://github.com/dlmacedo/entropic-out-of-distribution-detection.\n