Adversarial Concurrent Training: Optimizing Robustness and Accuracy Trade-off of Deep Neural Networks

Adversarial training has been proven to be an effective technique for\nimproving the adversarial robustness of models. However, there seems to be an\ninherent trade-off between optimizing the model for accuracy and robustness. To\nthis end, we propose Adversarial Concurrent Training (ACT), which employs\nadversarial training in a collaborative learning framework whereby we train a\nrobust model in conjunction with a natural model in a minimax game. ACT\nencourages the two models to align their feature space by using the\ntask-specific decision boundaries and explore the input space more broadly.\nFurthermore, the natural model acts as a regularizer, enforcing priors on\nfeatures that the robust model should learn. Our analyses on the behavior of\nthe models show that ACT leads to a robust model with lower model complexity,\nhigher information compression in the learned representations, and high\nposterior entropy solutions indicative of convergence to a flatter minima. We\ndemonstrate the effectiveness of the proposed approach across different\ndatasets and network architectures. On ImageNet, ACT achieves 68.20% standard\naccuracy and 44.29% robustness accuracy under a 100-iteration untargeted\nattack, improving upon the standard adversarial training method's 65.70%\nstandard accuracy and 42.36% robustness.\n