Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalization

Adversarial examples cause neural networks to produce incorrect outputs with\nhigh confidence. Although adversarial training is one of the most effective\nforms of defense against adversarial examples, unfortunately, a large gap\nexists between test accuracy and training accuracy in adversarial training. In\nthis paper, we identify Adversarial Feature Overfitting (AFO), which may cause\npoor adversarially robust generalization, and we show that adversarial training\ncan overshoot the optimal point in terms of robust generalization, leading to\nAFO in our simple Gaussian model. Considering these theoretical results, we\npresent soft labeling as a solution to the AFO problem. Furthermore, we propose\nAdversarial Vertex mixup (AVmixup), a soft-labeled data augmentation approach\nfor improving adversarially robust generalization. We complement our\ntheoretical analysis with experiments on CIFAR10, CIFAR100, SVHN, and Tiny\nImageNet, and show that AVmixup significantly improves the robust\ngeneralization performance and that it reduces the trade-off between standard\naccuracy and adversarial robustness.\n