Utilizing a null class to restrict decision spaces and defend against neural network adversarial attacks

Despite recent progress, deep neural networks generally continue to be\nvulnerable to so-called adversarial examples--input images with small\nperturbations that can result in changes in the output classifications, despite\nno such change in the semantic meaning to human viewers. This is true even for\nseemingly simple challenges such as the MNIST digit classification task. In\npart, this suggests that these networks are not relying on the same set of\nobject features as humans use to make these classifications. In this paper we\nexamine an additional, and largely unexplored, cause behind this\nphenomenon--namely, the use of the conventional training paradigm in which the\nentire input space is parcellated among the training classes. Owing to this\nparadigm, learned decision spaces for individual classes span excessively large\nregions of the input space and include images that have no semantic similarity\nto images in the training set. In this study, we train models that include a\nnull class. That is, models may "opt-out" of classifying an input image as one\nof the digit classes. During training, null images are created through a\nvariety of methods, in an attempt to create tighter and more semantically\nmeaningful decision spaces for the digit classes. The best performing models\nclassify nearly all adversarial examples as nulls, rather than mistaking them\nas a member of an incorrect digit class, while simultaneously maintaining high\naccuracy on the unperturbed test set. The use of a null class and the training\nparadigm presented herein may provide an effective defense against adversarial\nattacks for some applications. Code for replicating this study will be made\navailable at https://github.com/mattroos/null_class_adversarial_defense .\n