Beyond cross-entropy: learning highly separable feature distributions for robust and accurate classification

Deep learning has shown outstanding performance in several applications\nincluding image classification. However, deep classifiers are known to be\nhighly vulnerable to adversarial attacks, in that a minor perturbation of the\ninput can easily lead to an error. Providing robustness to adversarial attacks\nis a very challenging task especially in problems involving a large number of\nclasses, as it typically comes at the expense of an accuracy decrease. In this\nwork, we propose the Gaussian class-conditional simplex (GCCS) loss: a novel\napproach for training deep robust multiclass classifiers that provides\nadversarial robustness while at the same time achieving or even surpassing the\nclassification accuracy of state-of-the-art methods. Differently from other\nframeworks, the proposed method learns a mapping of the input classes onto\ntarget distributions in a latent space such that the classes are linearly\nseparable. Instead of maximizing the likelihood of target labels for individual\nsamples, our objective function pushes the network to produce feature\ndistributions yielding high inter-class separation. The mean values of the\ndistributions are centered on the vertices of a simplex such that each class is\nat the same distance from every other class. We show that the regularization of\nthe latent space based on our approach yields excellent classification accuracy\nand inherently provides robustness to multiple adversarial attacks, both\ntargeted and untargeted, outperforming state-of-the-art approaches over\nchallenging datasets.\n